[lxc-users] Ubuntu 14.4 - unprivilaged nested containers - Failed to create hugetlb:lxc/test
Ivan Ogai
lxc-users at ogai.name
Thu Mar 31 13:52:50 UTC 2016
Hi Serge,
thanks for the good explanation below. I still have some open questions.
* Serge Hallyn <serge.hallyn at ubuntu.com> [2016-03-09 21:17]:
> Quoting Ivan Ogai (lxc-users at ogai.name):
> >
> > I repeat my last message but formatting it properly (sorry for the
> > original) and adding some info.
> >
> > I have a user 'jenkins' in a host running Ubuntu 14.04. The user is able
> > to create and start this unprivilaged container (also running Ubuntu
> > 14.04) whose config is:
> >
> > lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> > lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
> > lxc.arch = x86_64
> >
> > lxc.mount.auto = cgroup
> > lxc.aa_profile = lxc-container-default-with-nesting
> >
> > lxc.id_map = u 0 100000 65536
> > lxc.id_map = u 100000 165536 65536
> > lxc.id_map = g 0 100000 65536
> > lxc.id_map = g 100000 165536 65536
> > lxc.rootfs = /home/jenkins/.local/share/lxc/jenkins/rootfs
> > lxc.utsname = jenkins
> >
> > lxc.network.type = veth
> > lxc.network.flags = up
> > lxc.network.link = lxcbr0
> > lxc.network.hwaddr = 00:16:3e:17:02:1a
> >
> >
> > The idea is to use the ids in the host 165536-231072 for an unprivilaged
> > container inside the unprivilaged container above.
> >
> > Another user (also called jenkins) in the unprivilaged container jenkins
> > (with above config) is able to create unprivilaged (nested) containers
> > as expected, but is not able to start them. The log says:
> >
> > lxc-start 1457449899.746 INFO lxc_start_ui - lxc_start.c:main:264 - using rcfile /var/lib/jenkins/.local/share/lxc/test/config
> > lxc-start 1457449899.746 INFO lxc_confile - confile.c:config_idmap:1378 - read uid map: type u nsid 0 hostid 100000 range 65536
> > lxc-start 1457449899.746 INFO lxc_confile - confile.c:config_idmap:1378 - read uid map: type g nsid 0 hostid 100000 range 65536
> > lxc-start 1457449899.746 WARN lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized
> > lxc-start 1457449899.748 WARN lxc_cgmanager - cgmanager.c:cgm_get:985 - do_cgm_get exited with error
> > lxc-start 1457449899.748 INFO lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
> > lxc-start 1457449899.748 INFO lxc_seccomp - seccomp.c:use_seccomp:531 - Already seccomp-confined, not loading new policy
> > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/5' (5/6)
> > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/6' (7/8)
> > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/7' (9/10)
> > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/8' (11/12)
> > lxc-start 1457449899.748 INFO lxc_conf - conf.c:lxc_create_tty:3802 - tty's configured
> > lxc-start 1457449899.748 DEBUG lxc_start - start.c:setup_signal_fd:263 - sigchild handler set
> > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_peer_default:500 - opening /dev/tty for console peer
> > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_peer_default:506 - using '/dev/tty' as console
> > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_sigwinch_init:179 - 10902 got SIGWINCH fd 17
> > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_winsz:88 - set winsz dstfd:14 cols:382 rows:92
> > lxc-start 1457449899.931 INFO lxc_start - start.c:lxc_init:463 - 'test' is initialized
> > lxc-start 1457449899.931 DEBUG lxc_start - start.c:__lxc_start:1099 - Not dropping cap_sys_boot or watching utmp
> > lxc-start 1457449899.931 INFO lxc_start - start.c:lxc_spawn:832 - Cloning a new user namespace
> > lxc-start 1457449899.931 INFO lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgmanager initing for test
> > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:lxc_cgmanager_create:301 - call to cgmanager_create_sync failed: invalid request
> > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:lxc_cgmanager_create:303 - Failed to create hugetlb:lxc/test
> > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:cgm_create:650 - Error creating cgroup hugetlb:lxc/test
> > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: hugetlb:lxc/test did not exist
> > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: net_prio:lxc/test did not exist
> > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: perf_event:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: net_cls:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: freezer:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: devices:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: memory:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: blkio:lxc/test did not exist
> > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpuacct:lxc/test did not exist
> > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpu:lxc/test did not exist
> > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpuset:lxc/test did not exist
> > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: name=systemd:lxc/test did not exist
> > lxc-start 1457449899.935 ERROR lxc_start - start.c:lxc_spawn:891 - failed creating cgroups
> > lxc-start 1457449899.935 ERROR lxc_start - start.c:__lxc_start:1121 - failed to spawn 'test'
> > lxc-start 1457449899.935 ERROR lxc_start_ui - lxc_start.c:main:341 - The container failed to start.
> > lxc-start 1457449899.935 ERROR lxc_start_ui - lxc_start.c:main:345
> >
> > In the unprivilaged container jenkins as user jenkins, cat /proc/self/cgroup returns:
> >
> > 12:hugetlb:/user/1009.user/2.session/lxc/jenkins
>
> So /user/1009.user/2.session is the cgroup which was created on host for user
> jenkins. /user/1009.user/2.session/lxc/jenkins was created for that container.
> That is owned by root in that container.
Interesting. How can I see who owns which cgroup? I would have thought
that the cgroup indicated in `cat /proc/self/cgroup` is owned by the
user (self), not by root in the container.
If /user/1009.user/2.session/lxc/jenkins is owned by root, but the user
'jenkins' in the container needs to own something like
/user/1009.user/2.session/lxc/jenkins/user/1000.user/1.session
in order to create nested containers, how do I create that cgroup owned
by 'jenkins'?
Shouldn't the container just create that cgroup for a user when she
logs in the container, so that she can just create a nested container?
> Now to create containers in that
> container as user jenkins, you need to have a cgroup like
> /user/1009.user/2.session/lxc/jenkins/user/1000.user/1.session (or somesuch)
> owned by user jenkins in the container, so that it can create a
> /user/1009.user/2.session/lxc/jenkins/user/1000.user/1.session/lxc/jenkins
> cgroup for the container.
More information about the lxc-users
mailing list