[lxc-devel] Clone fails to start with overlayfslxc.mount.entry error

S.Çağlar Onur caglar at 10ur.org
Sat Aug 9 01:21:02 UTC 2014 

Hi David,

On Mon, Jul 28, 2014 at 10:32 AM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> Quoting David Richardson (pudnik019 at gmail.com):
>> Yes, I recall reading that the C API should be thread-safe, but I may not
>> be using the ref count primitives correctly?  At any rate, after cloning,
>> starting, stopping, and destroying a few thousand containers from multiple
>> concurrent threads, I eventually end up with a broken config file for a
>
> We do test for that in lxc-test-concurrent, and Çaglar does some large
> scale concurrency testing using the go api.  If we can see the code you're
> using that would be helpful.

[still catching emails so please ignore this if the problem gets resolved]

Tried doing a similar test using both C and Go but couldn't reproduce
this. Do you mind sharing the code or the exact set of operations that
you are doing?

>> container that looks like the following example:
>>
>> ---------------
>>
>> lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
>> lxc.mount.entry = sysfs sys sysfs defaults 0 0
>> lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
>> bind,optional 0 0
>> lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
>> lxc.mount.entry = /sys/kernel/security sys/kernel/security none
>> bind,optional 0 0
>> lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
>> lxc.tty = 4
>> lxc.pts = 1024
>> lxc.devttydir = lxc
>> lxc.arch = x86_64
>> lxc.cgroup.devices.deny = a
>> lxc.cgroup.devices.allow = c *:* m
>> lxc.cgroup.devices.allow = b *:* m
>> lxc.cgroup.devices.allow = c 1:3 rwm
>> lxc.cgroup.devices.allow = c 1:5 rwm
>> lxc.cgroup.devices.allow = c 5:0 rwm
>> lxc.cgroup.devices.allow = c 5:1 rwm
>> lxc.cgroup.devices.allow = c 1:8 rwm
>> lxc.cgroup.devices.allow = c 1:9 rwm
>> lxc.cgroup.devices.allow = c 5:2 rwm
>> lxc.cgroup.devices.allow = c 136:* rwm
>> lxc.cgroup.devices.allow = c 254:0 rm
>> lxc.cgroup.devices.allow = c 10:229 rwm
>> lxc.cgroup.devices.allow = c 10:200 rwm
>> lxc.cgroup.devices.allow = c 1:7 rwm
>> lxc.cgroup.devices.allow = c 10:228 rwm
>> lxc.cgroup.devices.allow = c 10:232 rwm
>> lxc.utsname = qpJ14Klr8KvP987h
>> lxc.network.type = veth
>> lxc.network.flags = up
>> lxc.network.link = lxcbr0
>> lxc.network.hwaddr = 00:16:3e:04:04:04
>> lxc.cap.drop = sys_module
>> lxc.cap.drop = mac_admin
>> lxc.cap.drop = mac_override
>> lxc.cap.drop = sys_time
>> lxc.rootfs = overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid
>> 0 0
>> lxc.mount.entry = sysfs sys sysfs defaults 0 0
>> lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
>> bind,optional 0 0
>> lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
>> lxc.mount.entry = /sys/kernel/security sys/kernel/security none
>> bind,optional 0 0
>> lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
>> lxc.tty = 4
>> lxc.pts = 1024
>> lxc.devttydir = lxc
>> lxc.arch = x86_64
>> lxc.cgroup.devices.deny = a
>> lxc.cgroup.devices.allow = c *:* m
>> lxc.cgroup.devices.allow = b *:* m
>> lxc.cgroup.devices.allow = c 1:3 rwm
>> lxc.cgroup.devices.allow = c 1:5 rwm
>> lxc.cgroup.devices.allow = c 5:0 rwm
>> lxc.cgroup.devices.allow = c 5:1 rwm
>> lxc.cgroup.devices.allow = c 1:8 rwm
>> lxc.cgroup.devices.allow = c 1:9 rwm
>> lxc.cgroup.devices.allow = c 5:2 rwm
>> lxc.cgroup.devices.allow = c 136:* rwm
>> lxc.cgroup.devices.allow = c 254:0 rm
>> lxc.cgroup.devices.allow = c 10:229 rwm
>> lxc.cgroup.devices.allow = c 10:200 rwm
>> lxc.cgroup.devices.allow = c 1:7 rwm
>> lxc.cgroup.devices.allow = c 10:228 rwm
>> lxc.cgroup.devices.allow = c 10:232 rwm
>> lxc.utsname = ubuntu-14.04
>> lxc.network.type = veth
>> lxc.network.flags = up
>> lxc.network.link = lxcbr0
>> lxc.network.hwaddr = 00:16:3e:08:87:51
>> lxc.cap.drop = sys_module
>> lxc.cap.drop = mac_admin
>> lxc.cap.drop = mac_override
>> lxc.cap.drop = sys_time
>> lxc.pivotdir = lxc_putold
>>
>> ------------------
>>
>> You can see pretty clearly here where things go wrong: the substitution for
>> the rootfs.path into lxc.rootfs = PATH is getting garbled.  In fact, it
>> appears that it is substituting an entire copy of the parent's config file
>> (the clone's parent is named ubuntu-14.04) instead of the correct
>> overlayfs:/path/to/rootfs:/path/to/delta0.  I am running LXC 1.0.4 on an
>> Ubuntu-14.04 host.
>>
>> ~Dave
>>
>>
>> On Tue, Jul 22, 2014 at 9:10 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
>> wrote:
>>
>> > However if you're using the C api it is meant to be thread-safe.  Which
>> > version are you working with?  I'd love to see the program you're using,
>> > if you don't mind.
>> >
>> > Quoting David Richardson (pudnik019 at gmail.com):
>> > > Ah, yes, you are right.  Looks like the config file got garbled.  I
>> > suspect
>> > > it's because I am calling the C API's clone method from multiple threads
>> > in
>> > > a racey fashion.  I will look to ensure that any critical sections are
>> > > protected.  Thanks.
>> > >
>> > > ~Dave
>> > >
>> > >
>> > > On Tue, Jul 22, 2014 at 7:26 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
>> > > wrote:
>> > >
>> > > > Well, the configuration file is bad.
>> > > >
>> > > > overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
>> > > >
>> > > > is not valid.
>> > > >
>> > > > Please post the configuration file for the original and clone, as well
>> > as
>> > > > the exact way that you created the clone.
>> > > >
>> > > > -serge
>> > > > _______________________________________________
>> > > > lxc-devel mailing list
>> > > > lxc-devel at lists.linuxcontainers.org
>> > > > http://lists.linuxcontainers.org/listinfo/lxc-devel
>> > > >
>> >
>> > > _______________________________________________
>> > > lxc-devel mailing list
>> > > lxc-devel at lists.linuxcontainers.org
>> > > http://lists.linuxcontainers.org/listinfo/lxc-devel
>> >
>> > _______________________________________________
>> > lxc-devel mailing list
>> > lxc-devel at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-devel
>> >
>
>> _______________________________________________
>> lxc-devel mailing list
>> lxc-devel at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel



-- 
S.Çağlar Onur <caglar at 10ur.org>

More information about the lxc-devel mailing list