[lxc-users] systemd-based unprivileged containers
Christian Brauner
christianvanbrauner at gmail.com
Sun Dec 14 13:51:57 UTC 2014
Hello,
I'm using unprivileged lxc containers. Currently I'm trying to use
Debian Jessie. Which provides me with a few riddles. When I start the
container I get two error messages:
[chb at conventiont ~]$ lxc-start -n jessie -l DEBUG -o jessie
lxc-start: conf.c: mk_devtmpfs: 1318 Permission denied - Unable to
create /dev/.lxc for autodev
Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not
permitted
1) "lxc-start: conf.c: mk_devtmpfs: 1318 Permission denied - Unable to
create /dev/.lxc for autodev": I understand that unprivileged containers
do not have permissions to set up folders under /dev. Is there a
recommended way to solve this problem?
2) "Failed to mount cgroup at /sys/fs/cgroup/systemd: Operation not
permitted": What is goin on here and how can I solve this?
Here is the error log (container config files below):
lxc-start 1418564089.500 INFO lxc_start_ui - lxc_start.c:main:265 - using rcfile /home/chb/.local/share/lxc/jessie/config
lxc-start 1418564089.500 INFO lxc_confile - confile.c:config_idmap:1325 - read uid map: type u nsid 0 hostid 100000 range 65536
lxc-start 1418564089.500 INFO lxc_confile - confile.c:config_idmap:1325 - read uid map: type g nsid 0 hostid 100000 range 65536
lxc-start 1418564089.501 WARN lxc_log - log.c:lxc_log_init:316 - lxc_log_init called with log already initialized
lxc-start 1418564089.502 WARN lxc_cgmanager - cgmanager.c:cgm_get:954 - do_cgm_get exited with error
lxc-start 1418564089.503 DEBUG lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/2' (5/6)
lxc-start 1418564089.503 DEBUG lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/3' (7/8)
lxc-start 1418564089.503 DEBUG lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/4' (9/10)
lxc-start 1418564089.503 DEBUG lxc_conf - conf.c:lxc_create_tty:3665 - allocated pty '/dev/pts/5' (11/12)
lxc-start 1418564089.503 INFO lxc_conf - conf.c:lxc_create_tty:3676 - tty's configured
lxc-start 1418564089.503 DEBUG lxc_start - start.c:setup_signal_fd:247 - sigchild handler set
lxc-start 1418564089.503 DEBUG lxc_console - console.c:lxc_console_peer_default:500 - opening /dev/tty for console peer
lxc-start 1418564089.503 DEBUG lxc_console - console.c:lxc_console_peer_default:506 - using '/dev/tty' as console
lxc-start 1418564089.503 DEBUG lxc_console - console.c:lxc_console_sigwinch_init:179 - 2708 got SIGWINCH fd 17
lxc-start 1418564089.503 DEBUG lxc_console - console.c:lxc_console_winsz:88 - set winsz dstfd:14 cols:84 rows:49
lxc-start 1418564089.912 INFO lxc_start - start.c:lxc_init:443 - 'jessie' is initialized
lxc-start 1418564089.912 DEBUG lxc_start - start.c:__lxc_start:1058 - Not dropping cap_sys_boot or watching utmp
lxc-start 1418564089.912 INFO lxc_start - start.c:lxc_spawn:802 - Cloning a new user namespace
lxc-start 1418564089.912 INFO lxc_cgroup - cgroup.c:cgroup_init:62 - cgroup driver cgmanager initing for jessie
lxc-start 1418564090.110 NOTICE lxc_start - start.c:do_start:656 - switching to gid/uid 0 in new user namespace
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:setup_rootfs:1611 - mounted '/home/chb/.local/share/lxc/jessie/rootfs' on '/usr/lib/lxc/rootfs'
lxc-start 1418564090.116 INFO lxc_conf - conf.c:setup_utsname:900 - 'jessie' hostname has been setup
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:setup_hw_addr:2557 - mac address '00:16:3e:3a:f1:12' on 'eth0' has been setup
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:setup_netdev:2784 - 'eth0' has been setup
lxc-start 1418564090.116 INFO lxc_conf - conf.c:setup_network:2805 - network has been setup
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:check_autodev:3906 - Set exec command to /sbin/init
lxc-start 1418564090.116 INFO lxc_conf - conf.c:check_autodev:3920 - Container with systemd init detected - enabling autodev!
lxc-start 1418564090.116 INFO lxc_conf - conf.c:mount_autodev:1418 - Mounting /dev under /usr/lib/lxc/rootfs
lxc-start 1418564090.116 ERROR lxc_conf - conf.c:mk_devtmpfs:1318 - Permission denied - Unable to create /dev/.lxc for autodev
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:mount_check_fs:1250 - entering mount_check_fs for /home/chb/.local/share/lxc/jessie/rootfs.dev
lxc-start 1418564090.116 DEBUG lxc_conf - conf.c:mount_autodev:1449 - Mounting tmpfs to /home/chb/.local/share/lxc/jessie/rootfs.dev
lxc-start 1418564090.117 INFO lxc_conf - conf.c:mount_autodev:1476 - Mounted /dev under /usr/lib/lxc/rootfs
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted 'proc' on '/usr/lib/lxc/rootfs/proc', type 'proc'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted 'sysfs' on '/usr/lib/lxc/rootfs/sys', type 'sysfs'
lxc-start 1418564090.117 INFO lxc_conf - conf.c:mount_entry:2045 - failed to mount '/sys/fs/fuse/connections' on '/usr/lib/lxc/rootfs/sys/fs/fuse/connections' (optional): No such file or directory
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/console on /usr/lib/lxc/rootfs/dev/console to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/console was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/console' on '/usr/lib/lxc/rootfs/dev/console', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/full on /usr/lib/lxc/rootfs/dev/full to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/full was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/full' on '/usr/lib/lxc/rootfs/dev/full', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/null on /usr/lib/lxc/rootfs/dev/null to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/null was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/null' on '/usr/lib/lxc/rootfs/dev/null', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/random on /usr/lib/lxc/rootfs/dev/random to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/random was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/random' on '/usr/lib/lxc/rootfs/dev/random', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/tty on /usr/lib/lxc/rootfs/dev/tty to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/tty was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/tty' on '/usr/lib/lxc/rootfs/dev/tty', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/urandom on /usr/lib/lxc/rootfs/dev/urandom to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/urandom was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/urandom' on '/usr/lib/lxc/rootfs/dev/urandom', type 'none'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2056 - remounting /dev/zero on /usr/lib/lxc/rootfs/dev/zero to respect bind or remount options
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2071 - (at remount) flags for /dev/zero was 4098, required extra flags are 2
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:mount_entry:2106 - mounted '/dev/zero' on '/usr/lib/lxc/rootfs/dev/zero', type 'none'
lxc-start 1418564090.117 INFO lxc_conf - conf.c:mount_file_entries:2355 - mount points have been setup
lxc-start 1418564090.117 INFO lxc_conf - conf.c:setup_autodev:1504 - Creating initial consoles under /usr/lib/lxc/rootfs/dev
lxc-start 1418564090.117 INFO lxc_conf - conf.c:setup_autodev:1512 - Populating /dev under /usr/lib/lxc/rootfs
lxc-start 1418564090.117 INFO lxc_conf - conf.c:setup_autodev:1527 - Populated /dev under /usr/lib/lxc/rootfs
lxc-start 1418564090.117 INFO lxc_conf - conf.c:setup_dev_console:1836 - console has been setup
lxc-start 1418564090.117 INFO lxc_conf - conf.c:setup_tty:1027 - 4 tty(s) has been setup
lxc-start 1418564090.117 INFO lxc_conf - conf.c:do_tmp_proc_mount:3970 - I am 1, /proc/self points to '1'
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:setup_rootfs_pivot_root:1197 - created '/usr/lib/lxc/rootfs/lxc_putold' directory
lxc-start 1418564090.117 DEBUG lxc_conf - conf.c:setup_rootfs_pivot_root:1200 - mountpoint for old rootfs is '/usr/lib/lxc/rootfs/lxc_putold'
lxc-start 1418564090.118 DEBUG lxc_conf - conf.c:setup_rootfs_pivot_root:1213 - pivot_root syscall to '/usr/lib/lxc/rootfs' successful
lxc-start 1418564090.151 INFO lxc_conf - conf.c:umount_oldrootfs:1151 - lazy unmount of '/lxc_putold'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/dev'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/dev/shm'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/dev/pts'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/dev/hugepages'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/dev/mqueue'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/proc'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/proc/sys/fs/binfmt_misc'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/kernel/security'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/systemd'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/cpuset'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/cpu,cpuacct'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/memory'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/devices'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/freezer'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/net_cls'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/blkio'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/perf_event'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/cgroup/hugetlb'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/fs/pstore'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/firmware/efi/efivars'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/kernel/config'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/sys/kernel/debug'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/run'
lxc-start 1418564090.151 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/run/user/1000'
lxc-start 1418564090.152 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/tmp'
lxc-start 1418564090.152 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/boot'
lxc-start 1418564090.152 WARN lxc_conf - conf.c:umount_oldrootfs:1161 - failed to unmount '/lxc_putold/var/lib/docker/btrfs'
lxc-start 1418564090.152 INFO lxc_conf - conf.c:setup_personality:1791 - set personality to '0x0'
lxc-start 1418564090.152 NOTICE lxc_conf - conf.c:lxc_setup:4253 - 'jessie' is setup.
lxc-start 1418564090.152 NOTICE lxc_start - start.c:start:1152 - exec'ing '/sbin/init'
lxc-start 1418564090.153 NOTICE lxc_start - start.c:post_start:1163 - '/sbin/init' started with pid '2732'
lxc-start 1418564090.153 WARN lxc_start - start.c:signal_handler:295 - invalid pid for SIGCHLD
a) Container config file:
# Template used to create this container:
# /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d debian -r jessie -a amd64
# For additional config options, please look at lxc.container.conf(5)
# Distribution configuration
lxc.include = /usr/share/lxc/config/debian.common.conf
lxc.include = /usr/share/lxc/config/debian.userns.conf
lxc.arch = x86_64
# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /home/chb/.local/share/lxc/jessie/rootfs
lxc.utsname = jessie
# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = eth0
lxc.network.hwaddr = 00:16:3e:3a:f1:12
lxc.network.mtu = 1500
lxc.network.ipv4.gateway = 192.168.200.1
lxc.network.ipv4 = 192.168.200.12/24
b) /usr/share/lxc/debian.common.conf:
# Default pivot location
lxc.pivotdir = lxc_putold
# Default mount entries
lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0
lxc.mount.entry = sysfs sys sysfs defaults 0 0
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
bind,optional 0 0
# Default console settings
lxc.tty = 4
lxc.pts = 1024
# Default capabilities
lxc.cap.drop = sys_module mac_admin mac_override sys_time
# When using LXC with apparmor, the container will be confined by
# default.
# If you wish for it to instead run unconfined, copy the following line
# (uncommented) to the container's configuration file.
#lxc.aa_profile = unconfined
# To support container nesting on an Ubuntu host while retaining most of
# apparmor's added security, use the following two lines instead.
#lxc.aa_profile = lxc-container-default-with-nesting
#lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups
# If you wish to allow mounting block filesystems, then use the
# following
# line instead, and make sure to grant access to the block device and/or
# loop
# devices below in lxc.cgroup.devices.allow.
#lxc.aa_profile = lxc-container-default-with-mounting
# Default cgroup limits
lxc.cgroup.devices.deny = a
## Allow any mknod (but not using the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
## /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
## consoles
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 5:1 rwm
## /dev/{,u}random
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 1:9 rwm
## /dev/pts/*
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 136:* rwm
## rtc
lxc.cgroup.devices.allow = c 254:0 rm
## fuse
lxc.cgroup.devices.allow = c 10:229 rwm
## tun
lxc.cgroup.devices.allow = c 10:200 rwm
## full
lxc.cgroup.devices.allow = c 1:7 rwm
## hpet
lxc.cgroup.devices.allow = c 10:228 rwm
## kvm
lxc.cgroup.devices.allow = c 10:232 rwm
## To use loop devices, copy the following line to the container's
## configuration file (uncommented).
#lxc.cgroup.devices.allow = b 7:* rwm
# Blacklist some syscalls which are not safe in privileged
# containers
lxc.seccomp = /usr/share/lxc/config/common.seccomp
c) /usr/share/lxc/debian.userns.conf:
# CAP_SYS_ADMIN in init-user-ns is required for cgroup.devices
lxc.cgroup.devices.deny =
lxc.cgroup.devices.allow =
# Extra bind-mounts for userns
lxc.mount.entry = /dev/console dev/console none bind,create=file 0 0
lxc.mount.entry = /dev/full dev/full none bind,create=file 0 0
lxc.mount.entry = /dev/null dev/null none bind,create=file 0 0
lxc.mount.entry = /dev/random dev/random none bind,create=file 0 0
lxc.mount.entry = /dev/tty dev/tty none bind,create=file 0 0
lxc.mount.entry = /dev/urandom dev/urandom none bind,create=file 0 0
lxc.mount.entry = /dev/zero dev/zero none bind,create=file 0 0
# Default seccomp policy is not needed for unprivileged containers, and
# non-root users cannot use seccmp without NNP anyway.
lxc.seccomp =
Best,
Christian
More information about the lxc-users
mailing list