[lxc-devel] Error "unshare: Operation not permitted" when trying to create user container

Thu Feb 20 06:52:59 UTC 2014

On Feb 20, 2014, at 1:29 AM, Brian Campbell <lambda at continuation.org> wrote:

> On Feb 18, 2014, at 10:25 AM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> 
>> Quoting Brian Campbell (lambda at continuation.org):
>>> On Feb 18, 2014, at 12:16 AM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
>>> 
>>>>> Ah, that's the ticket:
>>>>> 
>>>>> lambda at gherkin:~$ cat /proc/sys/kernel/unprivileged_userns_clone
>>>>> 0
>>>>> 
>>>>> Looks like this is a Debian specific patch,
>>>> 
>>>> *cough* pls not to ask how i knew to query it kthx
>>>> 
>>>>> which is why looking at the upstream kernel source left me puzzled about why I'd be getting an EPERM.
>>>>> 
>>>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=712870
>>>>> 
>>>>> * namespaces: Enable USER_NS (Closes: #712870)
>>>>>   - Restrict creation of user namespaces to root (CAP_SYS_ADMIN) by
>>>>>     default (sysctl: kernel.unprivileged_userns_clone)
>>>>> 
>>>>> Works much better when I flip that to 1!
>>>>> 
>>>>>  lambda at gherkin:lxc$ lxc-create -l DEBUG -o lxc.log --name precise-test -t download -- -d ubuntu -r precise -a amd64Setting up the GPG keyring
>>>>>  Downloading the image index
>>>>>  Downloading the rootfs
>>>>>  Downloading the metadata
>>>>>  The image cache is now ready
>>>>>  Unpacking the rootfs
>>>>> 
>>>>>  ---
>>>>>  You just created an Ubuntu container (release=precise, arch=amd64).
>>>>>  The default username/password is: ubuntu / ubuntu
>>>>>  To gain root privileges, please use sudo.
>>>>> 
>>>>> Now I need to figure out what is required for the setup of cgroups,
>>>>> since now that's failing. It looks like it's trying to clear out the
>>>>> cgroup hierarchy to be able to set it up differently, but obviously
>>>>> doesn't have permissions to do so. I'm running systemd, which uses the
>>>>> cgroup hierarchy already. I've seen references to cgroup-lite,
>>>>> cgroup-bin, and cgroup-tools; do I need one of these to packages to
>>>>> set up cgroups appropriately for unprivileged containers? Or is it
>>>>> possible to do natively with systemd?
>>>>> 
>>>>> lambda at gherkin:lxc$ lxc-start -n precise-test
>>>>> lxc_container: Could not set clone_children to 1 for cpuset hierarchy in parent cgroup.
>>>> 
>>>> I thought we'd stopped doing that, but I guess not fully.
>>>> Could you try this patch?
>>>> 
>>>> Subject: [PATCH 1/1] continue if we cannot set cpuset.clonechildren
>>>> 
>>>> Signed-off-by: Serge Hallyn <serge.hallyn at ubuntu.com>
>>> 
>>> It does get rid of that specific error, but still goes on to fail:
>>> 
>>> lambda at gherkin:lxc (master)$ lxc-start -n precise-test
>>> lxc_container: Permission denied - Could not create cgroup /precise-test
>>> lxc_container: Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/
>> 
>> It looks like you're in the root cgroup and starting as non-root.
>> Without being root you indeed do not have the rights to create new
>> cgroups there.  You'll need to either use lxc as root, or do something
>> like
>> 
>> for d in /sys/fs/cgroup/*; do
>> 	sudo mkdir $d/lambda
>> 	sudo chown -R lambda: $d/lambda
>> 	echo $$ > $d/lambda/tasks
>> done
> 
> 
> Apologies for the slow followup, been a busy few days.
> 
> Doing that gives me an error on the the cpuset cgroup (added an echo to see which one it was):
> 
> /sys/fs/cgroup/blkio/lambda
> /sys/fs/cgroup/cpu/lambda
> /sys/fs/cgroup/cpuacct/lambda
> /sys/fs/cgroup/cpu,cpuacct/lambda
> /sys/fs/cgroup/cpuset/lambda
> -bash: echo: write error: No space left on device
> /sys/fs/cgroup/devices/lambda
> /sys/fs/cgroup/freezer/lambda
> /sys/fs/cgroup/net_cls/lambda
> /sys/fs/cgroup/perf_event/lambda
> /sys/fs/cgroup/systemd/lambda
> 
> I decided to see if it would work anyhow, but it still fails. Any clue why cpuset would be failing?

Ah, figured that out myself. Apparently you need to add cpus and mems to a cpuset cgroup before you can add any tasks to it. My question still stands about what normally handles this, so I can run that or at least reference the code for setting up my environment.

Doing that gets me another step closer, to a further error:

      lxc-start 1392878417.586 INFO     lxc_start_ui - using rcfile /home/lambda/.local/share/lxc/precise-test/config
      lxc-start 1392878417.586 INFO     lxc_confile - read uid map: type u nsid 0 hostid 100000 range 65536
      lxc-start 1392878417.586 INFO     lxc_confile - read uid map: type g nsid 0 hostid 100000 range 65536
      lxc-start 1392878417.586 WARN     lxc_log - lxc_log_init called with log already initialized
      lxc-start 1392878417.586 INFO     lxc_lsm - LSM security driver nop
      lxc-start 1392878417.586 DEBUG    lxc_conf - allocated pty '/dev/pts/3' (5/6)
      lxc-start 1392878417.586 DEBUG    lxc_conf - allocated pty '/dev/pts/4' (7/8)
      lxc-start 1392878417.586 DEBUG    lxc_conf - allocated pty '/dev/pts/5' (9/10)
      lxc-start 1392878417.586 DEBUG    lxc_conf - allocated pty '/dev/pts/6' (11/12)
      lxc-start 1392878417.586 INFO     lxc_conf - tty's configured
      lxc-start 1392878417.587 DEBUG    lxc_start - sigchild handler set
      lxc-start 1392878417.587 DEBUG    lxc_console - opening /dev/tty for console peer
      lxc-start 1392878417.587 INFO     lxc_caps - Last supported cap was 34
      lxc-start 1392878417.587 DEBUG    lxc_console - using '/dev/tty' as console
      lxc-start 1392878417.587 DEBUG    lxc_console - 21308 got SIGWINCH fd 17
      lxc-start 1392878417.587 DEBUG    lxc_console - set winsz dstfd:14 cols:161 rows:55
      lxc-start 1392878417.847 INFO     lxc_start - 'precise-test' is initialized
      lxc-start 1392878417.875 DEBUG    lxc_start - Not dropping cap_sys_boot or watching utmp
      lxc-start 1392878417.875 INFO     lxc_start - Cloning a new user namespace
      lxc-start 1392878417.875 INFO     lxc_cgroup - cgroup driver cgroupfs initing for precise-test
      lxc-start 1392878417.876 ERROR    lxc_cgfs - Operation not permitted - Could not add pid 21330 to cgroup /lambda/precise-test: internal error
      lxc-start 1392878417.909 ERROR    lxc_start - failed to spawn 'precise-test'

After changing that error to provide a little more information, I found that the full path is:

lxc-start: Operation not permitted - Could not add pid 23235 to cgroup /sys/fs/cgroup/devices/lambda/precise-test/tasks

-- Brian