[lxc-users] apparmor profile for systemd containers (WAS: Fedora container thinks it is not running)

[Fajar A. Nugraha]

On Fri, May 30, 2014 at 6:00 AM, Christian Seiler <christian at iwakd.de> wrote:
> Hi,
>
>> # lxc-attach -n f20 -- mount | grep cgroup
>> cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,size=12k,mode=755)
>> none on /sys/fs/cgroup/cgmanager type tmpfs (rw,relatime,size=4k,mode=755)
>> tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,mode=755)
>
> :-( This appears to be a rather nasty bug...

:D

The good news is that the workaroud (bind-mounting part of the systemd
cgroup mount manually) works with lxc.hook.mount, so at least we know
what systemd wants to be able to boot succesfully in a container.

> Hmm, are you running cgmanager at the same time as systemd? I think this

Yes. That is the default setup in ubuntu trusty.


> might be a problem with the intersection of cgmanager with the cgroup
> mounting code, i.e. the cgroup mounting code uses the cgfs stuff (which
> was originally just cgroup before Serge implemented multiple drivers)
> while the "put the container into cgroup" code uses cgmanager, which may
> have some weird side effect in this case. I have to confess that so far
> I haven't tried cgmanager myself (it's on my todo list), so I never
> tested the interaction between Serge's cgmanager code and my cgroup
> mounting code...
>
> If you are running cgmanager, could you try the same while cgmanager
> being stopped? Then LXC should fall back to the cgfs code, which
> *should* work in this case, unless something else broke this logic.

it kinda works now, but not the way you'd expect.
First it turns out I also needed to add lxc.mount.auto = sys before
lxc.mount.auto = cgroup:mixed (otherwise I'd get double /sys/fs/cgroup
tmpfs mount). Then after stopping cgmanager and starting the container
again, I get this

# mount | grep cgroup
cgroup_root on /sys/fs/cgroup type tmpfs (ro,relatime,size=10240k,mode=755)
systemd on /sys/fs/cgroup/systemd/user/0.user/13.session/lxc-all/f20
type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/run/cgmanager/agents/cgm-release-agent.systemd,name=systemd)
cgroup on /sys/fs/cgroup/hugetlb/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb)
cgroup on /sys/fs/cgroup/perf_event/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event)
cgroup on /sys/fs/cgroup/blkio/user/0.user/13.session/lxc-all/f20 type
cgroup (rw,relatime,blkio,release_agent=/run/cgmanager/agents/cgm-release-agent.blkio)
cgroup on /sys/fs/cgroup/freezer/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,freezer,release_agent=/run/cgmanager/agents/cgm-release-agent.freezer)
cgroup on /sys/fs/cgroup/devices/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,devices,release_agent=/run/cgmanager/agents/cgm-release-agent.devices)
cgroup on /sys/fs/cgroup/memory/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,memory,release_agent=/run/cgmanager/agents/cgm-release-agent.memory)
cgroup on /sys/fs/cgroup/cpuacct/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,cpuacct,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuacct)
cgroup on /sys/fs/cgroup/cpu/user/0.user/13.session/lxc-all/f20 type
cgroup (rw,relatime,cpu,release_agent=/run/cgmanager/agents/cgm-release-agent.cpu)
cgroup on /sys/fs/cgroup/cpuset/user/0.user/13.session/lxc-all/f20
type cgroup (rw,relatime,cpuset,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuset,clone_children)


What happpens is:
- the container still "Freezing execution" while starting root slice
- /sys/fs/cgroup/cpuset (and friends) are bind-mounted (there's
additional "user/0.user/13.session" directory, but I assume it's the
effect of the ubuntu hosts's systemd, and is okay)
- systemd mount in the container happens at
/sys/fs/cgroup/systemd/user/0.user/13.session/lxc-all/f20 , but the
container expects /sys/fs/cgroup/systemd/ to be writable

So lxc.mount.auto = cgroup:mixed and lxc.cgroup.use = @all works, but
it's not enough for fedora (and other sytemd-based container) to work
properly.

-- 
Fajar