[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie

Chris berzerkatives at gmail.com
Tue Sep 30 16:45:36 UTC 2014


On 30/09/14 16:47, Michael H. Warfield wrote:
> On Tue, 2014-09-30 at 15:46 +0100, Chris wrote:
>> On 29/09/14 21:46, Serge Hallyn wrote:
>>> Hm, sorry, not looking deeper right now, but :
>>>
>>>>         lxc-start 1411807327.953 ERROR    lxc_conf - Permission denied - WARNING: Failed to create symlink '/home/osmium/.local/share/lxc/osmium/rootfs.dev'->'/dev/.lxc/user/osmium.3c68b3f0c5eeec7d'
>>> Something will need to set that up.  I can't recall offhand
>>> what is supposed to do that.  Michael (cc:d), is that done
>>> through the init script?
>>>
>>> -serge
>>>
>>>
>> That might make sense, as I created this container through
>> debootstrapping the filesystem into
>> /home/osmium/.local/share/lxc/osmium/rootfs and then chown/grping all
>> the files to the appropriate users in this user's subuid/gid range...
>> pasted below in case anyone finds it useful. Please let me know if there
>> are further steps required to make this template/container valid.
> You created this with debootstrap?  So it's an Ubuntu or Debian
> container?  Why not use the appropriate lxc-create template?  They do a
> lot of things that you are unlikely to have done.  Since you're creating
> a container for an unprivileged user, you should probably have used the
> download template, as the live templates are generally for privileged
> users only.
I haven't looked a whole lot into the premade containers, my gut feeling 
was that I didn't want to download a whole operating system from this 
project, and that I'd be a lot more comfortable taking distribution that 
I trust, and making the template manually. This way I know everything 
extra that's going into it.
>
> That error is generated out of the code, which I authored, that sets up
> the autodev device areas and mounts that systemd mandates (but can still
> be used by anyone).  But, if this is Debian or Ubuntu, what version did
> you attempt to install?  Unless you're loading a test version, you
> shouldn't be getting systemd as your default init system manager (yet).
> If you have not explicitly set lxc.autodev = 1 in the config file and
> lxc-start does not detect systemd as the init system, you should not
> have ventured into that code at all.  I'm really baffled how you got in
> a situation where you used debootstrap and yet the code is running into
> the systemd autodev logic, something I would not have expected for
> Ubuntu or Debian just yet (and I don't think those templates are
> prepared to set up just yet).
It's running Debian Jessie. LXC 1.0.5-3 from package management. And 
systemd 208-8 also from package management.
>
> Next question...  How did you create your configuration file?  That
> error message is telling me that either you had lxc.autodev == 1 in the
> configuration file OR you're running systemd as your init system
> manager.  Neither of those should be a particular problem (well, systemd
> might if you haven't properly configured certain aspects of the unit
> files are startup - but you aren't getting that far) but it's just not
> clear how you got where you got doing what you did.
I took a config from an existing container and modified it for what I 
thought would work for an unprivileged container. I've attached the 
config for osmium. I've also attached the latest trace output from the 
lxc-start, as I've fixed a few slight errors in the config since then.
>
> What are the permissions on /home/osmium/.local/share/lxc/osmium ?  For
> some reason, lxc-start does not have permission to create a symlink in
> that directory (or maybe does not have rx read/search permission to all
> of its parent directories in the path).  That's a short-cut link back to
> the hash indexed dev directory under /dev/.lxc/user (for unpriv users)
> for the container /dev.  Creating that symlink depends only on the
> permissions in the path to the directory and the directory itself.
>
> Regards,
> Mike
>
>
osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium
drwxr-xr-x 3 osmium osmium 4096 Sep 30 15:38 
/home/osmium/.local/share/lxc/osmium
osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium/rootfs/
drwxr-xr-x 21 427680 427680 4096 Sep 14 15:56 
/home/osmium/.local/share/lxc/osmium/rootfs/
osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium/rootfs/dev
drwxr-xr-x 4 427680 427680 4096 Sep 14 15:56 
/home/osmium/.local/share/lxc/osmium/rootfs/dev

osmium at cadmium:~$ grep osmium /etc/sub[ug]id
/etc/subgid:osmium:427680:65536
/etc/subuid:osmium:427680:65536

osmium at cadmium:~$ find /dev/.lxc/user -ls
   9668    0 drwxrwxrwt   3 root     root           60 Sep 30 15:38 
/dev/.lxc/user
  11109    0 drwxr-xr-x   3 427680   427680         60 Sep 30 15:38 
/dev/.lxc/user/osmium.3c68b3f0c5eeec7d
  11110    0 drwxr-xr-x   2 427680   427680         40 Sep 30 15:38 
/dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts

Thanks,
Chris
-------------- next part --------------
# Container with network virtualized using a pre-configured bridge named br0 and
lxc.network.type = veth
#lxc.network.veth.pair = osmium
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:73:bd:de
lxc.id_map = u 0 427680 65536
lxc.id_map = g 0 427680 65536

# /var/lib/lxc/escher/config

## Container
lxc.utsname                             = osmium
lxc.rootfs                              = /home/osmium/.local/share/lxc/osmium/rootfs
lxc.arch                                = x86_64
lxc.console                             = /home/osmium/.console
lxc.tty                                 = 1
lxc.pts                                 = 1024

## Capabilities
lxc.cap.drop                            = mac_admin
lxc.cap.drop                            = mac_override
lxc.cap.drop                            = sys_admin
lxc.cap.drop                            = sys_module
## Devices
# Allow all devices
#lxc.cgroup.devices.allow               = a
# Deny all devices
lxc.cgroup.devices.deny                 = a
# Allow to mknod all devices (but not using them)
lxc.cgroup.devices.allow                = c *:* m
lxc.cgroup.devices.allow                = b *:* m

# /dev/console
lxc.cgroup.devices.allow                = c 5:1 rwm
# /dev/fuse
lxc.cgroup.devices.allow                = c 10:229 rwm
# /dev/null
lxc.cgroup.devices.allow                = c 1:3 rwm
# /dev/ptmx
lxc.cgroup.devices.allow                = c 5:2 rwm
# /dev/pts/*
lxc.cgroup.devices.allow                = c 136:* rwm
# /dev/random
lxc.cgroup.devices.allow                = c 1:8 rwm
# /dev/rtc
lxc.cgroup.devices.allow                = c 254:0 rwm
# /dev/tty
lxc.cgroup.devices.allow                = c 5:0 rwm
# /dev/urandom
lxc.cgroup.devices.allow                = c 1:9 rwm
# /dev/zero
lxc.cgroup.devices.allow                = c 1:5 rwm

## Limits
#lxc.cgroup.cpu.shares                  = 1024
#lxc.cgroup.cpuset.cpus                 = 0
#lxc.cgroup.memory.limit_in_bytes       = 256M
#lxc.cgroup.memory.memsw.limit_in_bytes = 1G

## Filesystem
lxc.mount.entry                         = proc /home/osmium/.local/share/lxc/osmium/rootfs/proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry				= devpts /home/osmium/.local/share/lxc/osmium/rootfs/dev/pts devpts defaults 0 0
lxc.mount.entry                         = sysfs /home/osmium/.local/share/lxc/osmium/rootfs/sys sysfs defaults,ro 0 0

-------------- next part --------------
      lxc-start 1412095368.928 INFO     lxc_start_ui - using rcfile /home/osmium/.local/share/lxc/osmium/config
      lxc-start 1412095368.928 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412095368.929 INFO     lxc_confile - read uid map: type u nsid 0 hostid 427680 range 65536
      lxc-start 1412095368.929 INFO     lxc_confile - read uid map: type g nsid 0 hostid 427680 range 65536
      lxc-start 1412095368.930 WARN     lxc_log - lxc_log_init called with log already initialized
      lxc-start 1412095368.930 INFO     lxc_lsm - LSM security driver nop
      lxc-start 1412095368.930 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412095368.931 DEBUG    lxc_conf - allocated pty '/dev/pts/1' (5/6)
      lxc-start 1412095368.931 INFO     lxc_conf - tty's configured
      lxc-start 1412095368.932 DEBUG    lxc_start - sigchild handler set
      lxc-start 1412095368.932 DEBUG    lxc_console - opening /home/osmium/.console for console peer
      lxc-start 1412095368.932 DEBUG    lxc_console - using '/home/osmium/.console' as console
      lxc-start 1412095368.932 DEBUG    lxc_console - no console peer
      lxc-start 1412095369.212 INFO     lxc_start - 'osmium' is initialized
      lxc-start 1412095369.243 DEBUG    lxc_start - Not dropping cap_sys_boot or watching utmp
      lxc-start 1412095369.243 INFO     lxc_start - Cloning a new user namespace
      lxc-start 1412095369.243 INFO     lxc_cgroup - cgroup driver cgroupfs initing for osmium
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.deny' set to 'a'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c *:* m'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'b *:* m'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:1 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 10:229 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:3 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:2 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 136:* rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:8 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 254:0 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 5:0 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:9 rwm'
      lxc-start 1412095369.247 DEBUG    lxc_cgfs - cgroup 'devices.allow' set to 'c 1:5 rwm'
      lxc-start 1412095369.247 INFO     lxc_cgfs - cgroup has been setup
      lxc-start 1412095369.310 NOTICE   lxc_start - switching to gid/uid 0 in new user namespace
      lxc-start 1412095369.313 DEBUG    lxc_conf - mounted '/home/osmium/.local/share/lxc/osmium/rootfs' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs'
      lxc-start 1412095369.314 INFO     lxc_conf - 'osmium' hostname has been setup
      lxc-start 1412095369.314 DEBUG    lxc_conf - mac address '00:16:3e:73:bd:de' on 'eth0' has been setup
      lxc-start 1412095369.315 DEBUG    lxc_conf - 'eth0' has been setup
      lxc-start 1412095369.315 INFO     lxc_conf - network has been setup
      lxc-start 1412095369.315 DEBUG    lxc_conf - Set exec command to /sbin/init
      lxc-start 1412095369.324 INFO     lxc_conf - Container with systemd init detected - enabling autodev!
      lxc-start 1412095369.324 INFO     lxc_conf - Mounting /dev under /usr/lib/x86_64-linux-gnu/lxc/rootfs
      lxc-start 1412095369.324 DEBUG    lxc_conf - entering mount_check_fs for /dev
      lxc-start 1412095369.325 DEBUG    lxc_conf - mount_check_fs returning 1 last devtmpfs
      lxc-start 1412095369.325 INFO     lxc_conf - Setup in /dev/.lxc failed.  Trying /dev/.lxc/user.
      lxc-start 1412095369.325 ERROR    lxc_conf - Permission denied - WARNING: Failed to create symlink '/home/osmium/.local/share/lxc/osmium/rootfs.dev'->'/dev/.lxc/user/osmium.3c68b3f0c5eeec7d'
      lxc-start 1412095369.325 DEBUG    lxc_conf - Bind mounting /dev/.lxc/user/osmium.3c68b3f0c5eeec7d to /usr/lib/x86_64-linux-gnu/lxc/rootfs/dev
      lxc-start 1412095369.325 INFO     lxc_conf - Mounted /dev under /usr/lib/x86_64-linux-gnu/lxc/rootfs
      lxc-start 1412095369.326 DEBUG    lxc_conf - mounted 'proc' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs//proc', type 'proc'
      lxc-start 1412095369.326 ERROR    lxc_conf - Invalid argument - failed to mount 'devpts' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs//dev/pts'
      lxc-start 1412095369.326 ERROR    lxc_conf - failed to setup the mount entries for 'osmium'
      lxc-start 1412095369.326 ERROR    lxc_start - failed to setup the container
      lxc-start 1412095369.326 ERROR    lxc_sync - invalid sequence number 1. expected 2
      lxc-start 1412095369.327 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412095369.419 ERROR    lxc_start - failed to spawn 'osmium'
      lxc-start 1412095369.420 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412095369.420 INFO     lxc_utils - XDG_RUNTIME_DIR isn't set in the environment.
      lxc-start 1412095369.421 ERROR    lxc_start_ui - The container failed to start.
      lxc-start 1412095369.421 ERROR    lxc_start_ui - Additional information can be obtained by setting the --logfile and --log-priority options.


More information about the lxc-users mailing list