[lxc-devel] problem with user namespace as root
Michael H. Warfield
mhw at WittsEnd.com
Thu Feb 13 20:19:20 UTC 2014
On Wed, 2014-02-12 at 20:18 +0100, Stephan Sachse wrote:
> i've lost my brain! i'm not a c programmer. someone who knows what
> he's doing, should have a look on it.
> it errors out with:
Crud... Looks like you started showing the info lines AFTER some of the
information I need would have been printed. Since it's the mount call
in setup_pts that's failing and giving you your first hard error, and
that's after we've pivoted to the rootfs of the container, I'm not sure
what the deal is with that "Invalid Argument". It may be possible that
the kernel is effectively not going to let us mount something on a
devtmpfs file system as a non-priv user at which point, the non-priv
user fallback in mk_devtmpfs is going to have to change to immediately
fall back to the tmpfs method in the non-priv case.
That first "Creating initial consoles" message is printed after all the
devtmpfs stuff is done. This sounds like something could be hosed back
further. There should have been some sort of INFO message about
"Mounting /dev under ..."
and maybe
"Setup in /dev/.lxc failed. Trying /dev/.lxc/user."
Those would have occurred before "Creating initial consoles". That
first message below would have been emitted from setup_autodev which is
called much further below in lxc_setup than mount_autodev is called. It
looks like the initial autodev setup did create a /dev for you. I'm
interesting in what may have been going on near the bottom of
mount_autodev just before the INFO line "Mounted /dev under %s". If you
saw that message, it would indicate that the /dev/pts directory should
have been created or already existed. If it fails there, you should see
an error about "Failed to create /dev/pts in container".
In that same region, prior to creating the /dev/pts directory, there
should have been one of three debug messages printed.
DEBUG("Bind mounting %s to %s", devtmpfs_path , path );
DEBUG("Mounting tmpfs to %s", host_path );
DEBUG("Bind mounting %s to %s", host_path, path );
(Yeah, I know, 2 of them are almost identical - it'll be the path
context.)
Knowing which path was taken through that logic could tell me a lot.
> lxc-start 1392232180.856 INFO lxc_conf - Creating initial consoles
> under /usr/lib64/lxc/rootfs/dev
> lxc-start 1392232180.856 INFO lxc_conf - Populating /dev under
> /usr/lib64/lxc/rootfs in_userns:1
> lxc-start 1392232180.856 INFO lxc_conf - Populated /dev under
> /usr/lib64/lxc/rootfs in_userns:1
> lxc-start 1392232180.856 INFO lxc_conf - created
> /usr/lib64/lxc/rootfs/dev/lxc
> lxc-start 1392232180.856 INFO lxc_conf - console has been setup on
> lxc/console
> lxc-start 1392232180.856 INFO lxc_conf - 4 tty(s) has been setup
> lxc-start 1392232180.856 DEBUG lxc_conf - created
> '/usr/lib64/lxc/rootfs/lxc_putold' directory
> lxc-start 1392232180.856 DEBUG lxc_conf - mountpoint for old rootfs
> is '/usr/lib64/lxc/rootfs/lxc_putold'
> lxc-start 1392232180.856 DEBUG lxc_conf - pivot_root syscall to
> '/usr/lib64/lxc/rootfs' successful
> lxc-start 1392232180.861 INFO lxc_conf - lazy unmount of '/lxc_putold'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount '/lxc_putold/dev'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/dev/pts'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/dev/shm'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/proc'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/proc/sys/fs/binfmt_misc'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount '/lxc_putold/sys'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/boot'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/media/dataa'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/cpuset'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/cpu'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/cpuacct'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/memory'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/devices'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/freezer'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/net_cls'
> lxc-start 1392232180.861 WARN lxc_conf - failed to unmount
> '/lxc_putold/cgroup/blkio'
> lxc-start 1392232180.861 ERROR lxc_conf - Invalid argument - failed
> to mount a new instance of '/dev/pts'
Without knowing what argument is invalid, it's really hard to guess
here. Since it works in the non-autodev case, I'm strongly suspecting
that the kernel is unhappy with us trying to mount that devpts onto
devtmpfs as a non-priv user which we are bind mounted to back in the
host.
As an experiment, you might try adding a "return NULL" in mk_devtmpfs
around line 1280 in conf.c right here:
--
--- conf.c.back 2014-02-13 14:14:04.753959574 -0500
+++ conf.c 2014-02-13 15:12:31.143644352 -0500
@@ -1278,6 +1278,10 @@
if ( 0 != access(tmp_path, F_OK) || 0 != stat(tmp_path, &s) || 0 == S_ISDIR(s.st_mode) ) {
ret = mkdir(tmp_path, S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH);
if ( ret ) {
+ /* Fall back immediately to tmpfs method */
+ INFO("Setup in /dev/.lxc failed. Falling back to tmpfs." );
+ return NULL;
+
/* Something must have failed with the base_path...
* Maybe unpriv user. Try user_path now... */
INFO("Setup in /dev/.lxc failed. Trying /dev/.lxc/user." );
--
If that change works, then the problem is that we cannot do that mount
of devpts on devtmpfs as a non-priv user in the host even with user
mapping to root in the container. Game over. We'll have to fall back
to using tmpfs in that case then and the rest of that branch will need
to be pulled.
> lxc-start 1392232180.861 ERROR lxc_conf - failed to setup the new
> pts instance
> lxc-start 1392232180.861 ERROR lxc_start - failed to setup the container
> lxc-start 1392232180.861 ERROR lxc_sync - invalid sequence number
> 1. expected 2
> lxc-start 1392232180.862 WARN lxc_conf - failed to remove interface '(null)'
> lxc-start 1392232180.878 ERROR lxc_start - failed to spawn 'fedora1'
>
> /stephan
>
> --
> Software is like sex, it's better when it's free!
I'm not currently setup to test the non-priv user stuff but I'll see
what I can do in the next couple of days. After that, I'm going to be
out-of-pocket for the next couple of weeks. I'll be able to respond to
E-Mail (and I'll be checking this list) but it's unlikely I'll be able
to work on much code.
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140213/f87e171a/attachment.pgp>
More information about the lxc-devel
mailing list