[lxc-devel] Clone fails to start with overlayfslxc.mount.entry error

Wed Jul 23 18:47:14 UTC 2014

Yes, I recall reading that the C API should be thread-safe, but I may not
be using the ref count primitives correctly?  At any rate, after cloning,
starting, stopping, and destroying a few thousand containers from multiple
concurrent threads, I eventually end up with a broken config file for a
container that looks like the following example:

---------------

lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
bind,optional 0 0
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none
bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.tty = 4
lxc.pts = 1024
lxc.devttydir = lxc
lxc.arch = x86_64
lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 254:0 rm
lxc.cgroup.devices.allow = c 10:229 rwm
lxc.cgroup.devices.allow = c 10:200 rwm
lxc.cgroup.devices.allow = c 1:7 rwm
lxc.cgroup.devices.allow = c 10:228 rwm
lxc.cgroup.devices.allow = c 10:232 rwm
lxc.utsname = qpJ14Klr8KvP987h
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:04:04:04
lxc.cap.drop = sys_module
lxc.cap.drop = mac_admin
lxc.cap.drop = mac_override
lxc.cap.drop = sys_time
lxc.rootfs = overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid
0 0
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
bind,optional 0 0
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none
bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.tty = 4
lxc.pts = 1024
lxc.devttydir = lxc
lxc.arch = x86_64
lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 254:0 rm
lxc.cgroup.devices.allow = c 10:229 rwm
lxc.cgroup.devices.allow = c 10:200 rwm
lxc.cgroup.devices.allow = c 1:7 rwm
lxc.cgroup.devices.allow = c 10:228 rwm
lxc.cgroup.devices.allow = c 10:232 rwm
lxc.utsname = ubuntu-14.04
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:08:87:51
lxc.cap.drop = sys_module
lxc.cap.drop = mac_admin
lxc.cap.drop = mac_override
lxc.cap.drop = sys_time
lxc.pivotdir = lxc_putold

------------------

You can see pretty clearly here where things go wrong: the substitution for
the rootfs.path into lxc.rootfs = PATH is getting garbled.  In fact, it
appears that it is substituting an entire copy of the parent's config file
(the clone's parent is named ubuntu-14.04) instead of the correct
overlayfs:/path/to/rootfs:/path/to/delta0.  I am running LXC 1.0.4 on an
Ubuntu-14.04 host.

~Dave

On Tue, Jul 22, 2014 at 9:10 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
wrote:

> However if you're using the C api it is meant to be thread-safe.  Which
> version are you working with?  I'd love to see the program you're using,
> if you don't mind.
>
> Quoting David Richardson (pudnik019 at gmail.com):
> > Ah, yes, you are right.  Looks like the config file got garbled.  I
> suspect
> > it's because I am calling the C API's clone method from multiple threads
> in
> > a racey fashion.  I will look to ensure that any critical sections are
> > protected.  Thanks.
> >
> > ~Dave
> >
> >
> > On Tue, Jul 22, 2014 at 7:26 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
> > wrote:
> >
> > > Well, the configuration file is bad.
> > >
> > > overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
> > >
> > > is not valid.
> > >
> > > Please post the configuration file for the original and clone, as well
> as
> > > the exact way that you created the clone.
> > >
> > > -serge
> > > _______________________________________________
> > > lxc-devel mailing list
> > > lxc-devel at lists.linuxcontainers.org
> > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> > >
>
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140723/57301723/attachment-0001.html>