[lxc-users] Ubuntu 14.4 - unprivilaged nested containers - Failed to create hugetlb:lxc/test
Serge Hallyn
serge.hallyn at ubuntu.com
Mon Apr 4 23:15:43 UTC 2016
Quoting Ivan Ogai (lxc-users at ogai.name):
> Hi Serge,
>
> thanks for the good explanation below. I still have some open questions.
>
> * Serge Hallyn <serge.hallyn at ubuntu.com> [2016-03-09 21:17]:
> > Quoting Ivan Ogai (lxc-users at ogai.name):
> > >
> > > I repeat my last message but formatting it properly (sorry for the
> > > original) and adding some info.
> > >
> > > I have a user 'jenkins' in a host running Ubuntu 14.04. The user is able
> > > to create and start this unprivilaged container (also running Ubuntu
> > > 14.04) whose config is:
> > >
> > > lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> > > lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
> > > lxc.arch = x86_64
> > >
> > > lxc.mount.auto = cgroup
> > > lxc.aa_profile = lxc-container-default-with-nesting
> > >
> > > lxc.id_map = u 0 100000 65536
> > > lxc.id_map = u 100000 165536 65536
> > > lxc.id_map = g 0 100000 65536
> > > lxc.id_map = g 100000 165536 65536
> > > lxc.rootfs = /home/jenkins/.local/share/lxc/jenkins/rootfs
> > > lxc.utsname = jenkins
> > >
> > > lxc.network.type = veth
> > > lxc.network.flags = up
> > > lxc.network.link = lxcbr0
> > > lxc.network.hwaddr = 00:16:3e:17:02:1a
> > >
> > >
> > > The idea is to use the ids in the host 165536-231072 for an unprivilaged
> > > container inside the unprivilaged container above.
> > >
> > > Another user (also called jenkins) in the unprivilaged container jenkins
> > > (with above config) is able to create unprivilaged (nested) containers
> > > as expected, but is not able to start them. The log says:
> > >
> > > lxc-start 1457449899.746 INFO lxc_start_ui - lxc_start.c:main:264 - using rcfile /var/lib/jenkins/.local/share/lxc/test/config
> > > lxc-start 1457449899.746 INFO lxc_confile - confile.c:config_idmap:1378 - read uid map: type u nsid 0 hostid 100000 range 65536
> > > lxc-start 1457449899.746 INFO lxc_confile - confile.c:config_idmap:1378 - read uid map: type g nsid 0 hostid 100000 range 65536
> > > lxc-start 1457449899.746 WARN lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized
> > > lxc-start 1457449899.748 WARN lxc_cgmanager - cgmanager.c:cgm_get:985 - do_cgm_get exited with error
> > > lxc-start 1457449899.748 INFO lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
> > > lxc-start 1457449899.748 INFO lxc_seccomp - seccomp.c:use_seccomp:531 - Already seccomp-confined, not loading new policy
> > > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/5' (5/6)
> > > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/6' (7/8)
> > > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/7' (9/10)
> > > lxc-start 1457449899.748 DEBUG lxc_conf - conf.c:lxc_create_tty:3791 - allocated pty '/dev/pts/8' (11/12)
> > > lxc-start 1457449899.748 INFO lxc_conf - conf.c:lxc_create_tty:3802 - tty's configured
> > > lxc-start 1457449899.748 DEBUG lxc_start - start.c:setup_signal_fd:263 - sigchild handler set
> > > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_peer_default:500 - opening /dev/tty for console peer
> > > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_peer_default:506 - using '/dev/tty' as console
> > > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_sigwinch_init:179 - 10902 got SIGWINCH fd 17
> > > lxc-start 1457449899.748 DEBUG lxc_console - console.c:lxc_console_winsz:88 - set winsz dstfd:14 cols:382 rows:92
> > > lxc-start 1457449899.931 INFO lxc_start - start.c:lxc_init:463 - 'test' is initialized
> > > lxc-start 1457449899.931 DEBUG lxc_start - start.c:__lxc_start:1099 - Not dropping cap_sys_boot or watching utmp
> > > lxc-start 1457449899.931 INFO lxc_start - start.c:lxc_spawn:832 - Cloning a new user namespace
> > > lxc-start 1457449899.931 INFO lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgmanager initing for test
> > > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:lxc_cgmanager_create:301 - call to cgmanager_create_sync failed: invalid request
> > > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:lxc_cgmanager_create:303 - Failed to create hugetlb:lxc/test
> > > lxc-start 1457449899.932 ERROR lxc_cgmanager - cgmanager.c:cgm_create:650 - Error creating cgroup hugetlb:lxc/test
> > > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: hugetlb:lxc/test did not exist
> > > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: net_prio:lxc/test did not exist
> > > lxc-start 1457449899.933 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: perf_event:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: net_cls:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: freezer:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: devices:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: memory:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: blkio:lxc/test did not exist
> > > lxc-start 1457449899.934 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpuacct:lxc/test did not exist
> > > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpu:lxc/test did not exist
> > > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: cpuset:lxc/test did not exist
> > > lxc-start 1457449899.935 INFO lxc_cgmanager - cgmanager.c:cgm_remove_cgroup:528 - cgroup removal attempt: name=systemd:lxc/test did not exist
> > > lxc-start 1457449899.935 ERROR lxc_start - start.c:lxc_spawn:891 - failed creating cgroups
> > > lxc-start 1457449899.935 ERROR lxc_start - start.c:__lxc_start:1121 - failed to spawn 'test'
> > > lxc-start 1457449899.935 ERROR lxc_start_ui - lxc_start.c:main:341 - The container failed to start.
> > > lxc-start 1457449899.935 ERROR lxc_start_ui - lxc_start.c:main:345
> > >
> > > In the unprivilaged container jenkins as user jenkins, cat /proc/self/cgroup returns:
> > >
> > > 12:hugetlb:/user/1009.user/2.session/lxc/jenkins
> >
> > So /user/1009.user/2.session is the cgroup which was created on host for user
> > jenkins. /user/1009.user/2.session/lxc/jenkins was created for that container.
> > That is owned by root in that container.
>
> Interesting. How can I see who owns which cgroup? I would have thought
ls -l. But by convention 1009.user suggests that uid 1009 will own it...
> that the cgroup indicated in `cat /proc/self/cgroup` is owned by the
> user (self), not by root in the container.
In order for the user to own the cgroup, root has to chown the cgroup
to the user.
> If /user/1009.user/2.session/lxc/jenkins is owned by root, but the user
> 'jenkins' in the container needs to own something like
The last piece of that path, 'jenkins', refers to the container name,
not the user name.
> /user/1009.user/2.session/lxc/jenkins/user/1000.user/1.session
>
> in order to create nested containers, how do I create that cgroup owned
> by 'jenkins'?
Edit /etc/pam.d/common-session, change the libpam-cgfs line to add 'blkio'
to the list of controllers (or remove the '-c controllerlist' arguments
altogether to do it for all controllers)
> Shouldn't the container just create that cgroup for a user when she
> logs in the container, so that she can just create a nested container?
A blkio cgroup is not strictly required to create a container. (Older
lxc requires it, newer does not).
More information about the lxc-users
mailing list