[lxc-users] Can't start unprivileged container in Ubuntu 14.04 with LXC 2
Ben Warren
ben at skyportsystems.com
Tue May 9 16:40:33 UTC 2017
> On May 9, 2017, at 8:10 AM, Serge E. Hallyn <serge at hallyn.com> wrote:
>
<snip>
>
>>
>> I’ve made some progress, but still don’t fully know what’s going on. When I build lxc from source (top-of-tree github.com:lxc/lxc) and compile with full cgmanager and libcap support, the generated binaries work, and I can start not only my ‘trusty’ container, but also ones that are farther from the host, such as ‘delian-stretch’, which is systemd-based.
>>
>> The difference I see in the log is which cgroup driver is used.
>> When I build using the binaries from ’trusty-backports’, I see this:
>> lxc-start 20170509054154.989 INFO lxc_cgroup - cgroups/cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for cd-build
>>
>> When using the binaries I built from source, I see this:
>> lxc-start 20170509053256.861 INFO lxc_cgroup - cgroups/cgroup.c:cgroup_init:68 - cgroup driver cgmanager initing for cd-build
>>
>> Assuming cgmanager support is compiled in to the ‘trusty-backports’ version, the following code determines if the cgmanager driver is used (non-NULL return code means cgmanager is to be used):
>>
>> struct cgroup_ops *cgm_ops_init(void)
>> {
>> check_supports_multiple_controllers(-1);
>> if (!collect_subsystems())
>> return NULL;
>>
>> if (api_version < CGM_SUPPORTS_MULT_CONTROLLERS)
>> cgm_all_controllers_same = false;
>>
>> // if root, try to escape to root cgroup
>> if (geteuid() == 0 && !cgm_escape(NULL)) {
>> free_subsystems();
>> return NULL;
>> }
>>
>> return &cgmanager_ops;
>> }
>>
>> I have no context for how any of this is dependent on the environment, although I’m sure you do :)
>
> Mine were starting with cgfsng which yours is using also, so you don't *need*
> the cgmanager driver. But I'm pretty sure that if you build your own with
> it enabled it will work.
>
> Is it possible that you have lxc.cgroup.use set in /etc/lxc/lxc.conf or in
> ~/.config/lxc/lxc.conf, and that it includes 'cpu'? If so, assuming you
> don't need it, removing cpu should work around this failure.
>
Neither of these files is present. This is it for config:
ben at ben-sc:~/tmp/lxc/src$ cat /etc/lxc/default.conf
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:xx:xx:xx
ben at ben-sc:~/tmp/lxc/src$ cat ~/.config/lxc/default.conf
lxc.id_map = u 0 165536 65536
lxc.id_map = g 0 165536 65536
> Does adding ',cpu" to the end of the pam_cgfs.so line in /etc/pam.d/common-session
> help?
>
I added like this:
session optional pam_cgfs.so -c freezer,memory,cpu,name=systemd
but it doesn’t seem to make a difference
> The other thing is back to your core problem - why is /sys/fs/cgroup/cpu not
> remountable read-only? It may be related to why you have a dsystemd cgroup
> hierarchy. Do you recall setting that up and/or why it's there? Can you
> show the contents of /proc/1/mounts and /proc/self/mounts on the host and a
> fresh host boot log?
I think the dsystemd thing was left over from me trying something else. It’s not there now, after reverting to before any LXC installation and just installing the backports version of lxc.
Here’s the current state. If I run ‘lxc-start’ runtime-linked against the ‘back ports’ shared libraries I get this message:
lxc-start 20170509161114.691 INFO lxc_conf - conf.c:mount_file_entries:1985 - mount points have been setup
lxc-start 20170509161114.691 ERROR lxc_cgfsng - cgroups/cgfsng.c:do_secondstage_mounts_if_needed:1557 - Operation not permitted - Error remounting /usr/lib/x86_64-linux-gnu/lxc/sys/fs/cgroup/cpuset read-only
lxc-start 20170509161114.691 ERROR lxc_conf - conf.c:lxc_mount_auto_mounts:839 - Operation not permitted - error mounting /sys/fs/cgroup
If I change LD_LIBRARY_PATH to use the .so that I built, the container start as previously mentioned, using cgmanager.
ben at ben-sc:~$ cat /proc/self/cgroup
11:name=systemd:/user/1001.user/c2.session
10:perf_event:/user/1001.user/c2.session
9:memory:/user/1001.user/c2.session
8:hugetlb:/user/1001.user/c2.session
7:freezer:/user/1001.user/c2.session
6:devices:/user/1001.user/c2.session
5:cpuacct:/user/1001.user/c2.session
4:blkio:/user/1001.user/c2.session
3:cpu:/user/1001.user/c2.session
2:cpuset:/user/1001.user/c2.session
ben at ben-sc:~$ cat /proc/1/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=4073948k,nr_inodes=1018487,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=816968k,mode=755 0 0
/dev/disk/by-uuid/0fdaee58-1394-4338-9eed-95ab207f0de6 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
none /sys/fs/cgroup tmpfs rw,relatime,size=4k,mode=755 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
none /run/user tmpfs rw,nosuid,nodev,noexec,relatime,size=102400k,mode=755 0 0
none /sys/fs/pstore pstore rw,relatime 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset,clone_children 0 0
cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
cgmfs /run/cgmanager/fs tmpfs rw,relatime,size=100k,mode=755 0 0
cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory,release_agent=/run/cgmanager/agents/cgm-release-agent.memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices,release_agent=/run/cgmanager/agents/cgm-release-agent.devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer,release_agent=/run/cgmanager/agents/cgm-release-agent.freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio,release_agent=/run/cgmanager/agents/cgm-release-agent.blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb 0 0
name=systemd /sys/fs/cgroup/systemd cgroup rw,relatime,release_agent=/run/cgmanager/agents/cgm-release-agent.systemd,name=systemd 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
lxcfs /var/lib/lxcfs fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
gvfsd-fuse /run/user/1001/gvfs fuse.gvfsd-fuse rw,nosuid,nodev,relatime,user_id=1001,group_id=1001 0 0
ben at ben-sc:~$ cat /proc/self/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=4073948k,nr_inodes=1018487,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=816968k,mode=755 0 0
/dev/disk/by-uuid/0fdaee58-1394-4338-9eed-95ab207f0de6 / ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
none /sys/fs/cgroup tmpfs rw,relatime,size=4k,mode=755 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
none /run/user tmpfs rw,nosuid,nodev,noexec,relatime,size=102400k,mode=755 0 0
none /sys/fs/pstore pstore rw,relatime 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset,clone_children 0 0
cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
cgmfs /run/cgmanager/fs tmpfs rw,relatime,size=100k,mode=755 0 0
cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct,release_agent=/run/cgmanager/agents/cgm-release-agent.cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory,release_agent=/run/cgmanager/agents/cgm-release-agent.memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices,release_agent=/run/cgmanager/agents/cgm-release-agent.devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer,release_agent=/run/cgmanager/agents/cgm-release-agent.freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio,release_agent=/run/cgmanager/agents/cgm-release-agent.blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb 0 0
name=systemd /sys/fs/cgroup/systemd cgroup rw,relatime,release_agent=/run/cgmanager/agents/cgm-release-agent.systemd,name=systemd 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
lxcfs /var/lib/lxcfs fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
gvfsd-fuse /run/user/1001/gvfs fuse.gvfsd-fuse rw,nosuid,nodev,relatime,user_id=1001,group_id=1001 0 0
More information about the lxc-users
mailing list