[lxc-devel] [RFC PATCH 0/1] allow setting cgroup items before the cgroup is entered

[Dwight Engen]

On Wed, 15 May 2013 10:46:13 -0500
Serge Hallyn <serge.hallyn at ubuntu.com> wrote:

> Quoting Dwight Engen (dwight.engen at oracle.com):
> > On Tue, 14 May 2013 11:01:04 -0500
> > Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> > 
> > > Quoting Dwight Engen (dwight.engen at oracle.com):
> > > > Hi,
> > > > 
> > > > I tried to put lxc.cgroup.memory.kmem.limit_in_bytes = 4194304
> > > > in a config file to test forkbomb prevention. The problem with
> > > > this is that kmem.limit_in_bytes (per the kernel documentation)
> > > > "cannot be set if the cgroup have children, or if there are
> > > > already tasks in the cgroup". Currently, lxc does
> > > > lxc_cgroup_enter() before doing setup_cgroup() in the
> > > > lxc_spawn() flow and therefore gets -EBUSY.
> > > > 
> > > > Commit 544a48a0 leads me to believe that we don't want to move
> > > > setup_cgroup() earlier, so I've refactored it to create an
> > > > additional setup_cgroup_pre_enter() that gets called before
> > > > lxc_cgroup_enter(). Currently it only writes
> > > > kmem.limit_in_bytes, if there are other items that need to be
> > > > setup pre-enter, they could easily be added to the list. This
> > > > fixes the problem for me. I think we are trying to keep lxc
> > > > from knowing about specific cgroup control knobs, but I have
> > > > not thought of another way to do this. Thoughts?
> > > 
> > > I think the only problem with configuring cgroups early is that
> > > some block devices which the container might want to mount could
> > > be denied.
> > 
> > Hi Serge, I think by this you mean having lxc mount a block device
> > into the container that the container won't have a
> > cgroup.devices.allow line for. So I tried putting a line like:
> > 
> >   lxc.mount.entry = /dev/sda1 mnt none defaults 0 0  
> 
> Can you try specifying the actual fstype instead of none?

Doh! Thanks, yep none was the problem. I initially started with auto,
but that wasn't working (doesn't work in mount syscall, only with the
mount program). We should probably document that you need to specify the
fs type in lxc.conf(5) where it talks about lxc.mount. I can send a
patch.

> Of course also make sure that your config has
> 
> lxc.cgroup.devices.allow = b 8:1 rwm

The current code works either with or without this since we don't
setup devices cgroup till after the mounts are done.

> > in the conf, but this did not work for me even with the current code
> > (setup_cgroup() after lxc_cgroup_enter(), and more importantly after
> > the child has done lxc_setup() and done the mounts). The child is
> > getting "No such device" so I'm confused as to the use case we're
> > trying to make work. Probably I'm simply doing something wrong here
> > to get it mounted.
> > 
> > > So perhaps we should do the inverse of what you're doing.
> > > Configure all cgroups right before lxc_cgroup_enter(), except for
> > > devices cgroup, which gets configured after mounts happen?
> > 
> > This is easy to do, but I'd like a test case to try out :)
> > 
> > > -serge