[lxc-devel] problem with user namespace as root
Stéphane Graber
stgraber at ubuntu.com
Wed Feb 12 15:17:19 UTC 2014
On Wed, Feb 12, 2014 at 02:58:54PM +0100, Stephan Sachse wrote:
> os: centos-6.5
> kernel: 3.13.2-2.el6.x86_64 (handcrafted rpm)
> lxc: 1.0.0-0.4.3.beta4.el6.x86_64 (handcrafted rpm from git ce79774)
>
> lxc-create -n fedora1 -t fedora -- -R 20
> lxc-start -n fedora1 -d
> lxc-console -n fedora1
> lxc-stop -n fedora1
>
> all works fine :)
>
> # ls -l /var/lib/lxc/fedora1/
> -rw-r--r-- 1 root root 1118 12. Feb 12:10 config
> dr-xr-xr-x 18 root root 4096 12. Feb 12:15 rootfs
> lrwxrwxrwx 1 root root 34 12. Feb 12:15 rootfs.dev ->
> /dev/.lxc/fedora1.533098688727054a
>
> # ls -l /dev/.lxc/
> drwxr-xr-x 8 root root 540 12. Feb 12:15 fedora1.533098688727054a
> drwxrwxrwt 2 root root 40 12. Feb 12:12 user
>
> # ls -l /dev/.lxc/fedora1.533098688727054a
> drwxr-xr-x 2 root root 160 12. Feb 12:12 char
> lrwxrwxrwx 1 root root 11 12. Feb 12:15 console -> lxc/console
> lrwxrwxrwx 1 root root 11 12. Feb 12:12 core -> /proc/kcore
> lrwxrwxrwx 1 root root 13 12. Feb 12:12 fd -> /proc/self/fd
> crw-rw-rw- 1 root root 1, 7 12. Feb 12:15 full
> drwxr-xr-x 2 root root 40 12. Feb 12:12 hugepages
> prw------- 1 root root 0 12. Feb 12:12 initctl
> lrwxrwxrwx 1 root root 7 12. Feb 12:15 kmsg -> console
> drwxr-xr-x 2 root root 140 12. Feb 12:12 lxc
> drwxr-xr-x 2 root root 40 12. Feb 12:12 mqueue
> crw-rw-rw- 1 root root 1, 3 12. Feb 12:15 null
> lrwxrwxrwx 1 root root 13 12. Feb 12:12 ptmx -> /dev/pts/ptmx
> drwxr-xr-x 2 root root 40 12. Feb 12:12 pts
> crw-rw-rw- 1 root root 1, 8 12. Feb 12:15 random
> drwxr-xr-x 2 root root 40 12. Feb 12:12 shm
> lrwxrwxrwx 1 root root 15 12. Feb 12:12 stderr -> /proc/self/fd/2
> lrwxrwxrwx 1 root root 15 12. Feb 12:12 stdin -> /proc/self/fd/0
> lrwxrwxrwx 1 root root 15 12. Feb 12:12 stdout -> /proc/self/fd/1
> crw-rw-rw- 1 root tty 5, 0 12. Feb 12:15 tty
> lrwxrwxrwx 1 root root 8 12. Feb 12:15 tty1 -> lxc/tty1
> lrwxrwxrwx 1 root root 8 12. Feb 12:15 tty2 -> lxc/tty2
> lrwxrwxrwx 1 root root 8 12. Feb 12:15 tty3 -> lxc/tty3
> lrwxrwxrwx 1 root root 8 12. Feb 12:15 tty4 -> lxc/tty4
> crw-rw-rw- 1 root root 1, 9 12. Feb 12:15 urandom
> crw-rw-rw- 1 root root 1, 5 12. Feb 12:15 zero
>
> now reboot and see that /dev/.lxc is lost. but no problem, lxc-start
> -n fedora1 -d creates it again and all works fine.
>
> now aktivate the user namespace (map user root from container to uid
> 100000 on the host, all other as is)
>
> ~/uidmapshift -b /var/lib/lxc/fedora1/rootfs 0 100000 1
>
> edit /var/lib/lxc/fedora1/config and add
>
> lxc.id_map = u 0 100000 1
> lxc.id_map = g 0 100000 1
>
> try to start the container
>
> # lxc-start -l debug -o debug1.log -n fedora1
> lxc-start: File exists - WARNING: Failed to create symlink
> '/var/lib/lxc/fedora1/rootfs.dev'->'/dev/.lxc/
> fedora1.533098688727054a'
>
> lxc-start: Invalid argument - failed to mount a new instance of '/dev/pts'
> lxc-start: failed to setup the new pts instance
> lxc-start: failed to setup the container
> lxc-start: invalid sequence number 1. expected 2
> lxc-start: failed to spawn 'fedora1'
>
> the debug1.log is attached.
>
> now reboot. /dev/.lxc is lost again. try to start container fedora1
> with user namespace enabled
>
> # lxc-start -l debug -o debug2.log -n fedora1
> lxc-start: Permission denied - Unable to create /dev/.lxc for autodev
> lxc-start: Operation not permitted - Error creating null
>
> lxc-start: failed to populate /dev in the container
> lxc-start: failed to setup the container
> lxc-start: invalid sequence number 1. expected 2
> lxc-start: failed to spawn 'fedora1'
>
> the debug2.log attached
>
> my config for fedora1 is attache as config.txt
>
> where is the fault?
I suspect lxc.autodev is the problem, as far as I know (and the above
seems to prove it), it doesn't work with unprivileged containers as it
currently requires the ability to mknod.
>
> /stephan
>
> --
> Software is like sex, it's better when it's free!
> # Template used to create this container: /usr/share/lxc/templates/lxc-fedora
> # Parameters passed to the template: -R 20
> # For additional config options, please look at lxc.conf(5)
> lxc.network.type = veth
> lxc.network.flags = up
> lxc.network.link = lxcbr0
> lxc.network.hwaddr = fe:51:02:2c:73:fa
> lxc.rootfs = /var/lib/lxc/fedora1/rootfs
>
>
> lxc.devttydir = lxc
> lxc.tty = 4
> lxc.pts = 1024
>
> lxc.utsname = fedora1
> lxc.autodev = 1
>
> lxc.id_map = u 0 100000 1
> lxc.id_map = g 0 100000 1
>
> # systemctl start halt.target
> lxc.haltsignal = SIGRTMIN+3
>
> # Immediately halts the machine.
> lxc.stopsignal = SIGRTMIN+13
>
> lxc.hook.clone = /usr/share/lxc/hooks/clonehostname
>
> lxc.cap.drop = mac_admin mac_override
> lxc.cap.drop = setfcap
> lxc.cap.drop = sys_module sys_nice sys_pacct
> lxc.cap.drop = sys_rawio sys_time
>
> # Control Group devices: all denied except those whitelisted
> lxc.cgroup.devices.deny = a
> # Allow any mknod (but not reading/writing the node)
> lxc.cgroup.devices.allow = c *:* m
> lxc.cgroup.devices.allow = b *:* m
> # /dev/null
> lxc.cgroup.devices.allow = c 1:3 rwm
> # /dev/zero
> lxc.cgroup.devices.allow = c 1:5 rwm
> # /dev/full
> lxc.cgroup.devices.allow = c 1:7 rwm
> # /dev/tty
> lxc.cgroup.devices.allow = c 5:0 rwm
> # /dev/random
> lxc.cgroup.devices.allow = c 1:8 rwm
> # /dev/urandom
> lxc.cgroup.devices.allow = c 1:9 rwm
> # /dev/tty[1-4] ptys and lxc console
> lxc.cgroup.devices.allow = c 136:* rwm
> # /dev/ptmx pty master
> lxc.cgroup.devices.allow = c 5:2 rwm
>
> lxc-start 1392210686.191 INFO lxc_start_ui - using rcfile /var/lib/lxc/fedora1/config
> lxc-start 1392210686.191 INFO lxc_confile - read uid map: type u nsid 0 hostid 100000 range 1
> lxc-start 1392210686.191 INFO lxc_confile - read uid map: type g nsid 0 hostid 100000 range 1
> lxc-start 1392210686.191 WARN lxc_log - lxc_log_init called with log already initialized
> lxc-start 1392210686.192 DEBUG lxc_conf - allocated pty '/dev/pts/1' (5/6)
> lxc-start 1392210686.192 DEBUG lxc_conf - allocated pty '/dev/pts/2' (7/8)
> lxc-start 1392210686.192 DEBUG lxc_conf - allocated pty '/dev/pts/3' (9/10)
> lxc-start 1392210686.192 DEBUG lxc_conf - allocated pty '/dev/pts/4' (11/12)
> lxc-start 1392210686.192 INFO lxc_conf - tty's configured
> lxc-start 1392210686.192 DEBUG lxc_start - sigchild handler set
> lxc-start 1392210686.192 DEBUG lxc_console - opening /dev/tty for console peer
> lxc-start 1392210686.192 DEBUG lxc_console - using '/dev/tty' as console
> lxc-start 1392210686.192 DEBUG lxc_console - 3094 got SIGWINCH fd 17
> lxc-start 1392210686.192 DEBUG lxc_console - set winsz dstfd:14 cols:143 rows:42
> lxc-start 1392210686.192 INFO lxc_start - 'fedora1' is initialized
> lxc-start 1392210686.197 DEBUG lxc_start - Not dropping cap_sys_boot or watching utmp
>
> lxc-start 1392210686.197 INFO lxc_start - Cloning a new user namespace
> lxc-start 1392210686.198 DEBUG lxc_conf - instanciated veth 'veth6J2B2R/vethJT1MNW', index is '10'
> lxc-start 1392210686.198 INFO lxc_cgroup - cgroup driver cgroupfs initing for fedora1
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.deny' set to 'a'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c *:* m'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'b *:* m'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:3 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:5 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:7 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 5:0 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:8 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:9 rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 136:* rwm'
> lxc-start 1392210686.200 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 5:2 rwm'
> lxc-start 1392210686.200 INFO lxc_cgfs - cgroup has been setup
> lxc-start 1392210686.208 DEBUG lxc_conf - move '(null)' to '3100'
> lxc-start 1392210686.208 NOTICE lxc_start - switching to gid/uid 0 in new user namespace
> lxc-start 1392210686.208 INFO lxc_conf - 'fedora1' hostname has been setup
> lxc-start 1392210686.212 DEBUG lxc_conf - mac address 'fe:51:02:2c:73:fa' on 'eth0' has been setup
> lxc-start 1392210686.212 DEBUG lxc_conf - 'eth0' has been setup
> lxc-start 1392210686.212 INFO lxc_conf - network has been setup
> lxc-start 1392210686.215 DEBUG lxc_conf - mounted '/var/lib/lxc/fedora1/rootfs' on '/usr/lib64/lxc/rootfs'
> lxc-start 1392210686.215 INFO lxc_conf - Mounting /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392210686.215 DEBUG lxc_conf - entering mount_check_fs for /dev
>
> lxc-start 1392210686.215 DEBUG lxc_conf - mount_check_fs returning 1 last devtmpfs
>
> lxc-start 1392210686.215 ERROR lxc_conf - File exists - WARNING: Failed to create symlink '/var/lib/lxc/fedora1/rootfs.dev'->'/dev/.lxc/fedora1.533098688727054a'
>
> lxc-start 1392210686.215 DEBUG lxc_conf - Bind mounting /dev/.lxc/fedora1.533098688727054a to /usr/lib64/lxc/rootfs/dev
> lxc-start 1392210686.215 INFO lxc_conf - Mounted /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392210686.215 INFO lxc_conf - Creating initial consoles under /usr/lib64/lxc/rootfs/dev
>
> lxc-start 1392210686.215 INFO lxc_conf - Populating /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392210686.215 INFO lxc_conf - Populated /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392210686.215 INFO lxc_conf - created /usr/lib64/lxc/rootfs/dev/lxc
>
> lxc-start 1392210686.215 INFO lxc_conf - console has been setup on lxc/console
> lxc-start 1392210686.215 INFO lxc_conf - 4 tty(s) has been setup
> lxc-start 1392210686.215 DEBUG lxc_conf - created '/usr/lib64/lxc/rootfs/lxc_putold' directory
> lxc-start 1392210686.215 DEBUG lxc_conf - mountpoint for old rootfs is '/usr/lib64/lxc/rootfs/lxc_putold'
> lxc-start 1392210686.215 DEBUG lxc_conf - pivot_root syscall to '/usr/lib64/lxc/rootfs' successful
> lxc-start 1392210686.221 INFO lxc_conf - lazy unmount of '/lxc_putold'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/dev'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/dev/pts'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/dev/shm'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/proc'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/proc/sys/fs/binfmt_misc'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/sys'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/boot'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/media/dataa'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/cpuset'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/cpu'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/cpuacct'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/memory'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/devices'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/freezer'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/net_cls'
> lxc-start 1392210686.221 WARN lxc_conf - failed to unmount '/lxc_putold/cgroup/blkio'
> lxc-start 1392210686.221 ERROR lxc_conf - Invalid argument - failed to mount a new instance of '/dev/pts'
> lxc-start 1392210686.221 ERROR lxc_conf - failed to setup the new pts instance
> lxc-start 1392210686.221 ERROR lxc_start - failed to setup the container
> lxc-start 1392210686.221 ERROR lxc_sync - invalid sequence number 1. expected 2
> lxc-start 1392210686.221 WARN lxc_conf - failed to remove interface '(null)'
> lxc-start 1392210686.237 ERROR lxc_start - failed to spawn 'fedora1'
> lxc-start 1392211052.570 INFO lxc_start_ui - using rcfile /var/lib/lxc/fedora1/config
> lxc-start 1392211052.628 INFO lxc_confile - read uid map: type u nsid 0 hostid 100000 range 1
> lxc-start 1392211052.628 INFO lxc_confile - read uid map: type g nsid 0 hostid 100000 range 1
> lxc-start 1392211052.628 WARN lxc_log - lxc_log_init called with log already initialized
> lxc-start 1392211052.642 DEBUG lxc_conf - allocated pty '/dev/pts/1' (5/6)
> lxc-start 1392211052.642 DEBUG lxc_conf - allocated pty '/dev/pts/2' (7/8)
> lxc-start 1392211052.642 DEBUG lxc_conf - allocated pty '/dev/pts/3' (9/10)
> lxc-start 1392211052.642 DEBUG lxc_conf - allocated pty '/dev/pts/4' (11/12)
> lxc-start 1392211052.642 INFO lxc_conf - tty's configured
> lxc-start 1392211052.642 DEBUG lxc_start - sigchild handler set
> lxc-start 1392211052.642 DEBUG lxc_console - opening /dev/tty for console peer
> lxc-start 1392211052.642 DEBUG lxc_console - using '/dev/tty' as console
> lxc-start 1392211052.642 DEBUG lxc_console - 1204 got SIGWINCH fd 17
> lxc-start 1392211052.642 DEBUG lxc_console - set winsz dstfd:14 cols:143 rows:42
> lxc-start 1392211052.642 INFO lxc_start - 'fedora1' is initialized
> lxc-start 1392211052.653 DEBUG lxc_start - Not dropping cap_sys_boot or watching utmp
>
> lxc-start 1392211052.653 INFO lxc_start - Cloning a new user namespace
> lxc-start 1392211052.677 DEBUG lxc_conf - instanciated veth 'veth09P6F6/veth20PSS5', index is '4'
> lxc-start 1392211052.677 INFO lxc_cgroup - cgroup driver cgroupfs initing for fedora1
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.deny' set to 'a'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c *:* m'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'b *:* m'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:3 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:5 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:7 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 5:0 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:8 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 1:9 rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 136:* rwm'
> lxc-start 1392211052.678 DEBUG lxc_cgfs - cgroup 'devices.allow' set to 'c 5:2 rwm'
> lxc-start 1392211052.678 INFO lxc_cgfs - cgroup has been setup
> lxc-start 1392211052.686 DEBUG lxc_conf - move '(null)' to '1211'
> lxc-start 1392211052.686 NOTICE lxc_start - switching to gid/uid 0 in new user namespace
> lxc-start 1392211052.686 INFO lxc_conf - 'fedora1' hostname has been setup
> lxc-start 1392211052.693 DEBUG lxc_conf - mac address 'fe:51:02:2c:73:fa' on 'eth0' has been setup
> lxc-start 1392211052.693 DEBUG lxc_conf - 'eth0' has been setup
> lxc-start 1392211052.693 INFO lxc_conf - network has been setup
> lxc-start 1392211052.696 DEBUG lxc_conf - mounted '/var/lib/lxc/fedora1/rootfs' on '/usr/lib64/lxc/rootfs'
> lxc-start 1392211052.696 INFO lxc_conf - Mounting /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392211052.696 ERROR lxc_conf - Permission denied - Unable to create /dev/.lxc for autodev
> lxc-start 1392211052.696 DEBUG lxc_conf - entering mount_check_fs for /var/lib/lxc/fedora1/rootfs.dev
>
> lxc-start 1392211052.696 DEBUG lxc_conf - Mounting tmpfs to /var/lib/lxc/fedora1/rootfs.dev
> lxc-start 1392211052.730 INFO lxc_conf - Mounted /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392211052.730 INFO lxc_conf - Creating initial consoles under /usr/lib64/lxc/rootfs/dev
>
> lxc-start 1392211052.730 INFO lxc_conf - Populating /dev under /usr/lib64/lxc/rootfs
>
> lxc-start 1392211052.730 ERROR lxc_conf - Operation not permitted - Error creating null
>
> lxc-start 1392211052.730 ERROR lxc_conf - failed to populate /dev in the container
> lxc-start 1392211052.730 ERROR lxc_start - failed to setup the container
> lxc-start 1392211052.730 ERROR lxc_sync - invalid sequence number 1. expected 2
> lxc-start 1392211052.730 WARN lxc_conf - failed to remove interface '(null)'
> lxc-start 1392211052.745 ERROR lxc_start - failed to spawn 'fedora1'
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140212/9bb4f7dc/attachment-0001.pgp>
More information about the lxc-devel
mailing list