[lxc-devel] Clone fails to start with overlayfslxc.mount.entry error

Thu Aug 14 17:24:53 UTC 2014

I will get a small example coded up that repros the problem on my end, then
send it to you guys for review.  Thanks!

On Fri, Aug 8, 2014 at 6:21 PM, S.Çağlar Onur <caglar at 10ur.org> wrote:

> Hi David,
>
> On Mon, Jul 28, 2014 at 10:32 AM, Serge Hallyn <serge.hallyn at ubuntu.com>
> wrote:
> > Quoting David Richardson (pudnik019 at gmail.com):
> >> Yes, I recall reading that the C API should be thread-safe, but I may
> not
> >> be using the ref count primitives correctly?  At any rate, after
> cloning,
> >> starting, stopping, and destroying a few thousand containers from
> multiple
> >> concurrent threads, I eventually end up with a broken config file for a
> >
> > We do test for that in lxc-test-concurrent, and Çaglar does some large
> > scale concurrency testing using the go api.  If we can see the code
> you're
> > using that would be helpful.
>
> [still catching emails so please ignore this if the problem gets resolved]
>
> Tried doing a similar test using both C and Go but couldn't reproduce
> this. Do you mind sharing the code or the exact set of operations that
> you are doing?
>
> >> container that looks like the following example:
> >>
> >> ---------------
> >>
> >> lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
> >> lxc.mount.entry = sysfs sys sysfs defaults 0 0
> >> lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
> >> bind,optional 0 0
> >> lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional
> 0 0
> >> lxc.mount.entry = /sys/kernel/security sys/kernel/security none
> >> bind,optional 0 0
> >> lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
> >> lxc.tty = 4
> >> lxc.pts = 1024
> >> lxc.devttydir = lxc
> >> lxc.arch = x86_64
> >> lxc.cgroup.devices.deny = a
> >> lxc.cgroup.devices.allow = c *:* m
> >> lxc.cgroup.devices.allow = b *:* m
> >> lxc.cgroup.devices.allow = c 1:3 rwm
> >> lxc.cgroup.devices.allow = c 1:5 rwm
> >> lxc.cgroup.devices.allow = c 5:0 rwm
> >> lxc.cgroup.devices.allow = c 5:1 rwm
> >> lxc.cgroup.devices.allow = c 1:8 rwm
> >> lxc.cgroup.devices.allow = c 1:9 rwm
> >> lxc.cgroup.devices.allow = c 5:2 rwm
> >> lxc.cgroup.devices.allow = c 136:* rwm
> >> lxc.cgroup.devices.allow = c 254:0 rm
> >> lxc.cgroup.devices.allow = c 10:229 rwm
> >> lxc.cgroup.devices.allow = c 10:200 rwm
> >> lxc.cgroup.devices.allow = c 1:7 rwm
> >> lxc.cgroup.devices.allow = c 10:228 rwm
> >> lxc.cgroup.devices.allow = c 10:232 rwm
> >> lxc.utsname = qpJ14Klr8KvP987h
> >> lxc.network.type = veth
> >> lxc.network.flags = up
> >> lxc.network.link = lxcbr0
> >> lxc.network.hwaddr = 00:16:3e:04:04:04
> >> lxc.cap.drop = sys_module
> >> lxc.cap.drop = mac_admin
> >> lxc.cap.drop = mac_override
> >> lxc.cap.drop = sys_time
> >> lxc.rootfs = overlayfslxc.mount.entry = proc proc proc
> nodev,noexec,nosuid
> >> 0 0
> >> lxc.mount.entry = sysfs sys sysfs defaults 0 0
> >> lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
> >> bind,optional 0 0
> >> lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional
> 0 0
> >> lxc.mount.entry = /sys/kernel/security sys/kernel/security none
> >> bind,optional 0 0
> >> lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
> >> lxc.tty = 4
> >> lxc.pts = 1024
> >> lxc.devttydir = lxc
> >> lxc.arch = x86_64
> >> lxc.cgroup.devices.deny = a
> >> lxc.cgroup.devices.allow = c *:* m
> >> lxc.cgroup.devices.allow = b *:* m
> >> lxc.cgroup.devices.allow = c 1:3 rwm
> >> lxc.cgroup.devices.allow = c 1:5 rwm
> >> lxc.cgroup.devices.allow = c 5:0 rwm
> >> lxc.cgroup.devices.allow = c 5:1 rwm
> >> lxc.cgroup.devices.allow = c 1:8 rwm
> >> lxc.cgroup.devices.allow = c 1:9 rwm
> >> lxc.cgroup.devices.allow = c 5:2 rwm
> >> lxc.cgroup.devices.allow = c 136:* rwm
> >> lxc.cgroup.devices.allow = c 254:0 rm
> >> lxc.cgroup.devices.allow = c 10:229 rwm
> >> lxc.cgroup.devices.allow = c 10:200 rwm
> >> lxc.cgroup.devices.allow = c 1:7 rwm
> >> lxc.cgroup.devices.allow = c 10:228 rwm
> >> lxc.cgroup.devices.allow = c 10:232 rwm
> >> lxc.utsname = ubuntu-14.04
> >> lxc.network.type = veth
> >> lxc.network.flags = up
> >> lxc.network.link = lxcbr0
> >> lxc.network.hwaddr = 00:16:3e:08:87:51
> >> lxc.cap.drop = sys_module
> >> lxc.cap.drop = mac_admin
> >> lxc.cap.drop = mac_override
> >> lxc.cap.drop = sys_time
> >> lxc.pivotdir = lxc_putold
> >>
> >> ------------------
> >>
> >> You can see pretty clearly here where things go wrong: the substitution
> for
> >> the rootfs.path into lxc.rootfs = PATH is getting garbled.  In fact, it
> >> appears that it is substituting an entire copy of the parent's config
> file
> >> (the clone's parent is named ubuntu-14.04) instead of the correct
> >> overlayfs:/path/to/rootfs:/path/to/delta0.  I am running LXC 1.0.4 on an
> >> Ubuntu-14.04 host.
> >>
> >> ~Dave
> >>
> >>
> >> On Tue, Jul 22, 2014 at 9:10 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
> >> wrote:
> >>
> >> > However if you're using the C api it is meant to be thread-safe.
>  Which
> >> > version are you working with?  I'd love to see the program you're
> using,
> >> > if you don't mind.
> >> >
> >> > Quoting David Richardson (pudnik019 at gmail.com):
> >> > > Ah, yes, you are right.  Looks like the config file got garbled.  I
> >> > suspect
> >> > > it's because I am calling the C API's clone method from multiple
> threads
> >> > in
> >> > > a racey fashion.  I will look to ensure that any critical sections
> are
> >> > > protected.  Thanks.
> >> > >
> >> > > ~Dave
> >> > >
> >> > >
> >> > > On Tue, Jul 22, 2014 at 7:26 PM, Serge Hallyn <
> serge.hallyn at ubuntu.com>
> >> > > wrote:
> >> > >
> >> > > > Well, the configuration file is bad.
> >> > > >
> >> > > > overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
> >> > > >
> >> > > > is not valid.
> >> > > >
> >> > > > Please post the configuration file for the original and clone, as
> well
> >> > as
> >> > > > the exact way that you created the clone.
> >> > > >
> >> > > > -serge
> >> > > > _______________________________________________
> >> > > > lxc-devel mailing list
> >> > > > lxc-devel at lists.linuxcontainers.org
> >> > > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> > > >
> >> >
> >> > > _______________________________________________
> >> > > lxc-devel mailing list
> >> > > lxc-devel at lists.linuxcontainers.org
> >> > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> >
> >> > _______________________________________________
> >> > lxc-devel mailing list
> >> > lxc-devel at lists.linuxcontainers.org
> >> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> >
> >
> >> _______________________________________________
> >> lxc-devel mailing list
> >> lxc-devel at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-devel
> >
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
>
>
>
> --
> S.Çağlar Onur <caglar at 10ur.org>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140814/f3abe793/attachment.html>