<div dir="ltr">Yes, I recall reading that the C API should be thread-safe, but I may not be using the ref count primitives correctly? At any rate, after cloning, starting, stopping, and destroying a few thousand containers from multiple concurrent threads, I eventually end up with a broken config file for a container that looks like the following example:<div>
<br></div><div>---------------</div><div><br></div><div><div>lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0</div><div>lxc.mount.entry = sysfs sys sysfs defaults 0 0</div><div>lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0</div>
<div>lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0</div><div>lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0</div><div>lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0</div>
<div>lxc.tty = 4</div><div>lxc.pts = 1024</div><div>lxc.devttydir = lxc</div><div>lxc.arch = x86_64</div><div>lxc.cgroup.devices.deny = a</div><div>lxc.cgroup.devices.allow = c *:* m</div><div>lxc.cgroup.devices.allow = b *:* m</div>
<div>lxc.cgroup.devices.allow = c 1:3 rwm</div><div>lxc.cgroup.devices.allow = c 1:5 rwm</div><div>lxc.cgroup.devices.allow = c 5:0 rwm</div><div>lxc.cgroup.devices.allow = c 5:1 rwm</div><div>lxc.cgroup.devices.allow = c 1:8 rwm</div>
<div>lxc.cgroup.devices.allow = c 1:9 rwm</div><div>lxc.cgroup.devices.allow = c 5:2 rwm</div><div>lxc.cgroup.devices.allow = c 136:* rwm</div><div>lxc.cgroup.devices.allow = c 254:0 rm</div><div>lxc.cgroup.devices.allow = c 10:229 rwm</div>
<div>lxc.cgroup.devices.allow = c 10:200 rwm</div><div>lxc.cgroup.devices.allow = c 1:7 rwm</div><div>lxc.cgroup.devices.allow = c 10:228 rwm</div><div>lxc.cgroup.devices.allow = c 10:232 rwm</div><div>lxc.utsname = qpJ14Klr8KvP987h</div>
<div>lxc.network.type = veth</div><div>lxc.network.flags = up</div><div>lxc.network.link = lxcbr0</div><div>lxc.network.hwaddr = 00:16:3e:04:04:04</div><div>lxc.cap.drop = sys_module</div><div>lxc.cap.drop = mac_admin</div>
<div>lxc.cap.drop = mac_override</div><div>lxc.cap.drop = sys_time</div><div>lxc.rootfs = overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0</div><div>lxc.mount.entry = sysfs sys sysfs defaults 0 0</div><div>
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0</div><div>lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0</div><div>lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0</div>
<div>lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0</div><div>lxc.tty = 4</div><div>lxc.pts = 1024</div><div>lxc.devttydir = lxc</div><div>lxc.arch = x86_64</div><div>lxc.cgroup.devices.deny = a</div>
<div>lxc.cgroup.devices.allow = c *:* m</div><div>lxc.cgroup.devices.allow = b *:* m</div><div>lxc.cgroup.devices.allow = c 1:3 rwm</div><div>lxc.cgroup.devices.allow = c 1:5 rwm</div><div>lxc.cgroup.devices.allow = c 5:0 rwm</div>
<div>lxc.cgroup.devices.allow = c 5:1 rwm</div><div>lxc.cgroup.devices.allow = c 1:8 rwm</div><div>lxc.cgroup.devices.allow = c 1:9 rwm</div><div>lxc.cgroup.devices.allow = c 5:2 rwm</div><div>lxc.cgroup.devices.allow = c 136:* rwm</div>
<div>lxc.cgroup.devices.allow = c 254:0 rm</div><div>lxc.cgroup.devices.allow = c 10:229 rwm</div><div>lxc.cgroup.devices.allow = c 10:200 rwm</div><div>lxc.cgroup.devices.allow = c 1:7 rwm</div><div>lxc.cgroup.devices.allow = c 10:228 rwm</div>
<div>lxc.cgroup.devices.allow = c 10:232 rwm</div><div>lxc.utsname = ubuntu-14.04</div><div>lxc.network.type = veth</div><div>lxc.network.flags = up</div><div>lxc.network.link = lxcbr0</div><div>lxc.network.hwaddr = 00:16:3e:08:87:51</div>
<div>lxc.cap.drop = sys_module</div><div>lxc.cap.drop = mac_admin</div><div>lxc.cap.drop = mac_override</div><div>lxc.cap.drop = sys_time</div><div>lxc.pivotdir = lxc_putold</div></div><div><br></div><div>------------------</div>
<div><br></div><div>You can see pretty clearly here where things go wrong: the substitution for the rootfs.path into lxc.rootfs = PATH is getting garbled. In fact, it appears that it is substituting an entire copy of the parent's config file (the clone's parent is named ubuntu-14.04) instead of the correct overlayfs:/path/to/rootfs:/path/to/delta0. I am running LXC 1.0.4 on an Ubuntu-14.04 host.</div>
<div><br></div><div>~Dave</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jul 22, 2014 at 9:10 PM, Serge Hallyn <span dir="ltr"><<a href="mailto:serge.hallyn@ubuntu.com" target="_blank">serge.hallyn@ubuntu.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">However if you're using the C api it is meant to be thread-safe. Which<br>
version are you working with? I'd love to see the program you're using,<br>
if you don't mind.<br>
<div class="HOEnZb"><div class="h5"><br>
Quoting David Richardson (<a href="mailto:pudnik019@gmail.com">pudnik019@gmail.com</a>):<br>
> Ah, yes, you are right. Looks like the config file got garbled. I suspect<br>
> it's because I am calling the C API's clone method from multiple threads in<br>
> a racey fashion. I will look to ensure that any critical sections are<br>
> protected. Thanks.<br>
><br>
> ~Dave<br>
><br>
><br>
> On Tue, Jul 22, 2014 at 7:26 PM, Serge Hallyn <<a href="mailto:serge.hallyn@ubuntu.com">serge.hallyn@ubuntu.com</a>><br>
> wrote:<br>
><br>
> > Well, the configuration file is bad.<br>
> ><br>
> > overlayfslxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0<br>
> ><br>
> > is not valid.<br>
> ><br>
> > Please post the configuration file for the original and clone, as well as<br>
> > the exact way that you created the clone.<br>
> ><br>
> > -serge<br>
> > _______________________________________________<br>
> > lxc-devel mailing list<br>
> > <a href="mailto:lxc-devel@lists.linuxcontainers.org">lxc-devel@lists.linuxcontainers.org</a><br>
> > <a href="http://lists.linuxcontainers.org/listinfo/lxc-devel" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-devel</a><br>
> ><br>
<br>
> _______________________________________________<br>
> lxc-devel mailing list<br>
> <a href="mailto:lxc-devel@lists.linuxcontainers.org">lxc-devel@lists.linuxcontainers.org</a><br>
> <a href="http://lists.linuxcontainers.org/listinfo/lxc-devel" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-devel</a><br>
<br>
_______________________________________________<br>
lxc-devel mailing list<br>
<a href="mailto:lxc-devel@lists.linuxcontainers.org">lxc-devel@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-devel" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-devel</a><br>
</div></div></blockquote></div><br></div>