[lxc-users] Can't start unprivileged container in Ubuntu 14.04 with LXC 2

[Ben Warren]

> On May 9, 2017, at 1:34 PM, Serge E. Hallyn <serge at hallyn.com> wrote:
> 
> Hm, my last email (which I may have accidentally sent privately) was wrong,
> /proc/$$/mounts shows cgroups in fact mounted at /sys/fs/cgroup.
> However, take a look at this:
> 
>> cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset,clone_children 0 0
>> cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
> 
> These are different from the rest,
> 
That is strange.  I started from scratch again and this time only cpuset is not owned by cgmanager.
>> cgmfs /run/cgmanager/fs tmpfs rw,relatime,size=100k,mode=755 0 0
>> cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuacct 0 0
>> cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory,release_agent=/run/cgmanager/agents/cgm-release-agent.memory 0 0
>> cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices,release_agent=/run/cgmanager/agents/cgm-release-agent.devices 0 0
>> cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer,release_agent=/run/cgmanager/agents/cgm-release-agent.freezer 0 0
>> cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio,release_agent=/run/cgmanager/agents/cgm-release-agent.blkio 0 0
>> cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event 0 0
>> cgroup /sys/fs/cgroup/hugetlb cgroup rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb 0 0
>> name=systemd /sys/fs/cgroup/systemd cgroup rw,relatime,release_agent=/run/cgmanager/agents/cgm-release-agent.systemd,name=systemd 0 0
>> binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
> 
> So something is pre-mounting these filesystems before cgmanager starts.
> What mounted those?  I thought I'd asked for this before, but I don't
> see it in the thread - what does "dpkg -l | grep cgroup" show?  In
> particular I'm looking for cgroup-bin.
I don’t see that one.  It looks like cgroup-lite is installed, and I guess is responsible for the initial cgroup mounting?  How would you go about figuring out what process created the mounts?

ben at ben-sc:/etc/init$ dpkg -l | grep cgroup
ii  cgmanager                                             0.39-2ubuntu2~ubuntu14.04.1                         amd64        Central cgroup manager daemon
ii  cgroup-lite                                           1.11~ubuntu14.04.2                                  all          Light-weight package to set up cgroups at system boot
ii  libcgmanager-dev:amd64                                0.39-2ubuntu2~ubuntu14.04.1                         amd64        Central cgroup manager daemon (dev)
ii  libcgmanager0:amd64                                   0.39-2ubuntu2~ubuntu14.04.1                         amd64        Central cgroup manager daemon (client library)


It was installed as a prerequisite of lxc.  BTW - I’ve switched to using LXD since it seems to make management much easier, but since it still uses the same ‘liblxc1’ library, I have the same problem.

ben at ben-sc:~$ lxc launch ubuntu:14.04 u1
Creating u1
Starting u1                       
error: Error calling 'lxd forkstart u1 /var/lib/lxd/containers /var/log/lxd/u1/lxc.conf': err='exit status 1'
  lxc 20170510185615.517 ERROR lxc_cgfsng - cgroups/cgfsng.c:do_secondstage_mounts_if_needed:1557 - Operation not permitted - Error remounting /usr/lib/x86_64-linux-gnu/lxc/sys/fs/cgroup/cpuset read-only
  lxc 20170510185615.517 ERROR lxc_conf - conf.c:lxc_mount_auto_mounts:839 - Operation not permitted - error mounting /sys/fs/cgroup
  lxc 20170510185615.517 ERROR lxc_conf - conf.c:lxc_setup:3885 - failed to setup the automatic mounts for 'u1'

# Before installing anything:

ben at ben-sc:~$ cat /proc/$$/mounts | grep cgroup
none /sys/fs/cgroup tmpfs rw,relatime,size=4k,mode=755 0 0
systemd /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,name=systemd 0 0

As with LXC, if I build from source and use the generated ‘liblxc.so.1’ file, containers start fine but it uses the cgmanager driver.
I wonder why the packaged one uses the ‘cgfsng’ driver, even when cgmanager is installed and running?

—Ben