[lxc-devel] Containers do not start with lxc-1.0.0.beta2 on RHEL-6.5

[Robert Vogelgesang]

Hi Serge,

On Wed, Jan 22, 2014 at 08:39:13AM -0600, Serge Hallyn wrote:
> Quoting Robert Vogelgesang (vogel at users.sourceforge.net):
> > Hello Serge,
> > 
> > thank you for your clear explanation.
> > 
> > On Fri, Jan 17, 2014 at 04:21:49PM -0600, Serge Hallyn wrote:
> > [...]
> > > When you're not allowed to set clone_children, it is likely because
> > > there are already other child cgroups.  You cannot change clone_children
> > > in that case.
> > 
> > Sorry, no, this is not the case, there are no child cgroups.
> > 
> > I've done some more "research", and as it seems, the current RHEL-6
> > kernel, kernel-2.6.32-431.3.1.el6.x86_64, does not support the
> > cgroup.clone_children flag.  With this kernel, the /cgroup/cpuset/
> 
> Oh bother.
> 
> > directory does not contain a cgroup.clone_children file, and creating
> > it fails no matter how I try it.  If I read the kernel sources
> > correctly, the clone_children flag was introduced with kernel-2.6.37,
> > and Redhat didn't backport it to the RHEL-6 kernel.
> > 
> > > 
> > > When you get -ENOSPC it is because clone_children was not set, so the
> > > cpuset.mems and cpuset.cpus files were not initialized in the
> > > container's cgroups.  (I've always hated this behavior).
> > 
> > By using strace I've seen that lxc-start from lxc-0.9.0 simply ignores
> > the errors when setting cgroup.clone_children and this one when trying
> > to move the container into the cpuset cgroup; with lxc-0.9.0,
> > /cgroup/cpuset/lxc/test/tasks exists, but is empty.
> 
> It didn't simply ignore it, though, or it wouldn't be able to move tasks
> to it.  It manually initialized the cpuset.{cpus,mems} files for the
> new cgroups.

Sorry to contradict you, but lxc-0.9.0 did not initialize the
cpuset.{cpus,mems} files.  There is a comment in cgroup.c of lxc-0.9.0
marking this as a "XXX TODO" item.  Maybe older versions of lxc had
such code, but I didn't check.

As I wrote above, /cgroup/cpuset/lxc/test/tasks exists, but is empty
with lxc-0.9.0 when running the "test" container.

/cgroup/cpuset/lxc/test/cpuset.{cpus,mems} are both empty, too, with
a "stock" lxc-0.9.0 (from EPEL).


> 
> > Does this mean that with RHEL-6 one cannot use a cpuset cgroup, at
> > least not with lxc?  If so, is there a simple way I could tell the
> > lxc tools to leave it alone, even if it exists (i.e. is mounted)?
> 
> That won't suffice.  They *must* be initialized.

Yeah, in the meantime I found this statement in the Redhat Product
Documentation, too.

> 
> > > The *surest* way to avoid problems is to set up an early init job for
> > > yourself which sets clone_children to 1 in the root cpuset cgroup.
> > 
> > As I said above, I failed to set clone_children with the RHEL-6 kernel.
> > Is there some special trick or tool I must use to achieve this?
> 
> Nope.  Anyway we want to support kernels back to 2.6.32, so we do want
> this fixed.  If you're able to send (the simplest possible) patch to
> re-introduce the initialization of cgroups when they are created,
> please let me know.  Otherwise, please let me know and I will add it.
> 
> As I've said before noone is maintaining 0.9.0-stable, so the fix would
> be for 1.0.0, but the code should be very similar.

I'm not interested in an updated 0.9.0-stable, but in a working 1.0.

I'm currently developing a patch for 1.0.0.beta2 which should fix this,
but neither does it work ATM, nor am I sure that it is the simplest
possible patch. :-/

For the next two hours (or so), I'll try to fix the remaining problems.
If you think you can come up with a patch in shorter time, please
feel free to do so.

	Robert

> 
> -serge
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel