[lxc-devel] [RFC] [PATCH 0/6] Major cgroup logic rewrite

[Christian Seiler]

Hi all,

As discussed previously, I've now done a major rewrite of the entire
cgroup logic. There are now no assumptions made whatsoever when it
comes to the cgroup mount points, the kernel information will be used
to determine the proper locations for everything. (Only /proc is
assumed to be working properly.)

The patch series can also be found at my github under
<https://github.com/chris-se/lxc/tree/cgroup-major-rewrite>

I ended up almost completely rewriting cgroup.c to make it fit the new
scheme. For this reason, for the last patch for cgroup.[ch] it's
probably better to look at the new version and not the diff.

I did test the whole enchilada quite a bit, but I'm very sure that
there are a lot more bugs in there. What does work:

 - lxc-start (and -info, -stop)
 - Choosing $name-1, $name-2, ... as cgroup if $name is already used up
 - lxc-attach
 - lxc-freeze / lxc-unfreeze
 - lxc-stop -k on a frozen container
 - All subsystems mounted together in /sys/fs/cgroup directly
 - All subsystems mounted into different subdirectories beneath
   /sys/fs/cgroup
 - Although I didn't try actual nesting, on the systems with
   subsystems together in /sys/fs/cgroup I did:
      mkdir /sys/fs/cgroup/init /foo
      echo 1 > /sys/fs/cgroup/init/tasks
      mount --bind /sys/fs/cgroup/init /foo
      umount /sys/fs/cgroup
      mount --bind /foo /sys/fs/cgroup
      umount /foo
   And then lxc-start, which worked and correctly put the container
   in the /init/lxc/$name cgroup. This means that even with the current
   mountcgroups hook nesting should work (although as discussed before,
   the current mountcgroups logic breaks other software relying on
   cgroups [1])

Known issues:

 - Sometimes cgroup cleanup doesn't work 100%, sometimes cgroups are
   left behind. With the name collision avoidance this isn't a
   catastrophe, but it merits further investigation.
 - If cgroups.use in lxc.conf is set to something that expands to
   zero cgroups (because all selected controllers are not mounted,
   for example), currently it will fail at a weird place with a weird
   error message. This should either be caught (i.e. correct error
   message) or perhaps even allowed (i.e. no cgroups in that case)
 - Generally speaking, the global lxc.conf itself is not documented
   at all, currently.

Unknowns:

 - Bindings for anything other than Python, don't even know whether it
   compiles properly. (Python I didn't test either.)
 - Unit tests: They never worked on my system anyway, so I have no idea
   whether they still work.

Comments appreciated.

-- Christian

[1] I do want to get to the mountcgroups hook, just not today. ;-)