[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie

Tue Sep 30 22:56:09 UTC 2014

On 30/09/14 19:28, Michael H. Warfield wrote:
>> I haven't looked a whole lot into the premade containers, my gut feeling
>> was that I didn't want to download a whole operating system from this
>> project, and that I'd be a lot more comfortable taking distribution that
>> I trust, and making the template manually. This way I know everything
>> extra that's going into it.
> Our templates are pretty barebones.  Very minimal.  You'll have to add
> just about anything you would really want to make a useful container.
I should definitely take a closer look sometime.
>> It's running Debian Jessie. LXC 1.0.5-3 from package management. And
>> systemd 208-8 also from package management.
> OK... THAT explains a LOT!  That systemd option is why you're running
> into this problem and you're about to have far worse.
>
>> I took a config from an existing container and modified it for what I
>> thought would work for an unprivileged container. I've attached the
>> config for osmium. I've also attached the latest trace output from the
>> lxc-start, as I've fixed a few slight errors in the config since then.
> You're going to have to make some additional changes...  Make sure you
> add "lxc.kmsg = 0" to your container or systemd.journald is going to eat
> your CPU time for lunch (and be sure to flush
> your /dev/.lxc/user/osmium* directory).  There's also some adjustments
> that need to be made for mgetty consoles and such.  You also need to
> link the shutdown unit to the SIGPWR service to allow lxc to shut the
> container down gracefully.  You might take a look at the Oracle or
> Fedora templates for some guidance there.
Will definitely come back to this once it starts up, thank you for the 
advice.
>> osmium at cadmium:~$ find /dev/.lxc/user -ls
>>     9668    0 drwxrwxrwt   3 root     root           60 Sep 30 15:38
>> /dev/.lxc/user
>>    11109    0 drwxr-xr-x   3 427680   427680         60 Sep 30 15:38
>> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d
>>    11110    0 drwxr-xr-x   2 427680   427680         40 Sep 30 15:38
>> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts
> Bingo!
>
> Ok...  So it appears that lxc-start did manage to create your dev
> directory properly under the host /dev/.lxc/user.
>
> Now I see the real problem...
>
> The same code that creates that directory creates the symlink
> in /home/osmium/.local/share/lxc/osmium.  But, the /dev/ directory is
> owned by "427680:427680" while the directory containing the symlink is
> own by "osmium:osmium" and you then have a permission denied because
> 427680:427680 doesn't have write permissions
> to /home/osmium/.local/share/lxc/osmium.
>
> That's a (the!) problem.  I'm just not sure if chown/chgrp is the
> correct answer or if you need to add some group membership and add group
> write permissions with appropriate host auth secondary groups.  Either
> way, it's that permission problem that biting you in the rear end.
>
OK, yes. This was that problem. Fixing it has progressed startup a 
little further. It didn't like the lxc.mount.entry for devpts, so I 
threw that out for the time being also. Now it's still stuck at 
'populating dev' though. I've attached the latest trace in case you help 
me again.

osmium at cadmium:~$ lxc-start -n osmium -l trace -o /tmp/xxx7
lxc-start: Operation not permitted - Error creating null
lxc-start: failed to populate /dev in the container
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn 'osmium'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the 
--logfile and --log-priority options

Thanks,
Chris
-------------- next part --------------
      lxc-start 1412115865.294 INFO     lxc_start_ui - using rcfile /home/osmium/.local/share/lxc/osmium/config
      lxc-start 1412115865.294 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412115865.296 INFO     lxc_confile - read uid map: type u nsid 0 hostid 427680 range 65536
      lxc-start 1412115865.296 INFO     lxc_confile - read uid map: type g nsid 0 hostid 427680 range 65536
      lxc-start 1412115865.296 WARN     lxc_log - lxc_log_init called with log already initialized
      lxc-start 1412115865.296 INFO     lxc_lsm - LSM security driver nop
      lxc-start 1412115865.296 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412115865.298 DEBUG    lxc_conf - allocated pty '/dev/pts/1' (5/6)
      lxc-start 1412115865.298 INFO     lxc_conf - tty's configured
      lxc-start 1412115865.298 DEBUG    lxc_start - sigchild handler set
      lxc-start 1412115865.298 DEBUG    lxc_console - opening /home/osmium/.console for console peer
      lxc-start 1412115865.298 DEBUG    lxc_console - using '/home/osmium/.console' as console
      lxc-start 1412115865.298 DEBUG    lxc_console - no console peer
      lxc-start 1412115865.628 INFO     lxc_start - 'osmium' is initialized
      lxc-start 1412115865.659 DEBUG    lxc_start - Not dropping cap_sys_boot or watching utmp
      lxc-start 1412115865.659 INFO     lxc_start - Cloning a new user namespace
      lxc-start 1412115865.659 INFO     lxc_cgroup - cgroup driver cgroupfs initing for osmium
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.deny' set to 'a'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c *:* m'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'b *:* m'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:1 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 10:229 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:3 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:2 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 136:* rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:8 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 254:0 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:0 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:9 rwm'
      lxc-start 1412115865.663 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:5 rwm'
      lxc-start 1412115865.663 INFO     lxc_cgfs - cgroup has been setup
      lxc-start 1412115865.767 NOTICE   lxc_start - switching to gid/uid 0 in new user namespace
      lxc-start 1412115865.771 DEBUG    lxc_conf - mounted '/home/osmium/.local/share/lxc/osmium/rootfs' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs'
      lxc-start 1412115865.771 INFO     lxc_conf - 'osmium' hostname has been setup
      lxc-start 1412115865.772 DEBUG    lxc_conf - mac address '00:16:3e:73:bd:de' on 'eth0' has been setup
      lxc-start 1412115865.772 DEBUG    lxc_conf - 'eth0' has been setup
      lxc-start 1412115865.772 INFO     lxc_conf - network has been setup
      lxc-start 1412115865.772 DEBUG    lxc_conf - Set exec command to /sbin/init
      lxc-start 1412115865.772 INFO     lxc_conf - Container with systemd init detected - enabling autodev!
      lxc-start 1412115865.772 INFO     lxc_conf - Mounting /dev under /usr/lib/x86_64-linux-gnu/lxc/rootfs
      lxc-start 1412115865.772 DEBUG    lxc_conf - entering mount_check_fs for /dev
      lxc-start 1412115865.773 DEBUG    lxc_conf - mount_check_fs returning 1 last devtmpfs
      lxc-start 1412115865.773 INFO     lxc_conf - Setup in /dev/.lxc failed.  Trying /dev/.lxc/user.
      lxc-start 1412115865.773 DEBUG    lxc_conf - Bind mounting /dev/.lxc/user/osmium.3c68b3f0c5eeec7d to /usr/lib/x86_64-linux-gnu/lxc/rootfs/dev
      lxc-start 1412115865.773 INFO     lxc_conf - Mounted /dev under /usr/lib/x86_64-linux-gnu/lxc/rootfs
      lxc-start 1412115865.773 DEBUG    lxc_conf - mounted 'proc' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs//proc', type 'proc'
      lxc-start 1412115865.774 DEBUG    lxc_conf - mounted 'sysfs' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs//sys', type 'sysfs'
      lxc-start 1412115865.774 INFO     lxc_conf - mount points have been setup
      lxc-start 1412115865.774 INFO     lxc_conf - Creating initial consoles under /usr/lib/x86_64-linux-gnu/lxc/rootfs/dev
      lxc-start 1412115865.774 INFO     lxc_conf - Populating /dev under /usr/lib/x86_64-linux-gnu/lxc/rootfs
      lxc-start 1412115865.774 ERROR    lxc_conf - Operation not permitted - Error creating null
      lxc-start 1412115865.774 ERROR    lxc_conf - failed to populate /dev in the container
      lxc-start 1412115865.774 ERROR    lxc_start - failed to setup the container
      lxc-start 1412115865.774 ERROR    lxc_sync - invalid sequence number 1. expected 2
      lxc-start 1412115865.774 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412115865.835 ERROR    lxc_start - failed to spawn 'osmium'
      lxc-start 1412115865.836 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412115865.836 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412115865.837 ERROR    lxc_start_ui - The container failed to start.
      lxc-start 1412115865.837 ERROR    lxc_start_ui - Additional information can be obtained by setting the --logfile and --log-priority options.