[lxc-devel] nested containers

[Serge Hallyn]

Quoting S.Çağlar Onur (caglar at 10ur.org):
> Hi,
> 
> On Sat, Dec 28, 2013 at 5:21 AM, Stéphane Graber <stgraber at ubuntu.com> wrote:
> > On Fri, Dec 27, 2013 at 06:56:15PM -0500, S.Çağlar Onur wrote:
> >> Hey Pauk,
> >>
> >> On Fri, Dec 27, 2013 at 6:49 PM, Paul Wexler
> >> <paul at prometheusresearch.com> wrote:
> >> > Hello lxc community,
> >> >
> >> > Has anyone used nested containers?
> >> >
> >> > I am trying with limited success.  I cannot re-start them.
> >> > I can:
> >> >   1. create a container.
> >> >   2. configure it for nesting (I uncomment 2 lines in config).
> >> >   3. start the container.
> >> >   4. stop the container.
> >> >   5. but I cannot re-start the container.
> >> >
> >> > However, if I do not configure the container for nesting then
> >> > I can stop and re-start the container repeatedly without errors.
> >> >
> >> > Specifically, the following command line sequence fails on the
> >> > second lxc-start (please note I do not show the normal lxc-
> >> > output below, only the error msg):
> >> >
> >> >   # lxc-create -t ubuntu -n C00
> >> >   # X="lxc.aa_profile = lxc-container-default-with-nesting"
> >> >   # sed -i -e "s/^#$X/$X/" /var/lib/lxc/C00/config
> >> >   # X="lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups"
> >> >   # sed -i -e "s/^#$X/$X/" /var/lib/lxc/C00/config
> >> >   # lxc-start -d -n C00
> >> >   # lxc-stop -n C00
> >> >   # lxc-start -d -n C00
> >> >   lxc-start: command get_cgroup failed to receive response
> >>
> >> I believe this issue fixed after alpha has been released [1] so could
> >> you try replacing your mountcgroups hook with [2] and try again to see
> >> what will happen?
> >
> > Right, the issue here appears to be mountcgroups not cleaning up after
> > itself and being confused on the second run.
> 
> Oh, right I misunderstood the problem and though starts are failing
> due to lxc.include.
> 
> > Hopefully we'll be dropping that hook entirely with the introduction of
> > cgmanger in the next few weeks...
> >
> > Depending on what you are doing, you may also just comment that hook
> > entirely as cgroup-lite in the container will then simply mount the
> > cgroupfs controllers and LXC will be able to use them (however this will
> > most likely bypass any cgroup restriction you applied on the first
> > container, if any).
> 
> What about removing those leftover cgroup directories in post-stop hook?

That's probably a good idea.  Alternatively, lxc could try to set up a
release hook in each of the container's root cgroups (i.e.
'memory:/lxc/c1', 'cpuset:/lxc/c1', etc)

-serge