[lxc-devel] lxc-start: Invalid argument - pivot_root syscall failed

[Peter Volkov]

В Пт, 18/10/2013 в 00:22 +0400, Peter Volkov пишет:
> В Чт, 10/10/2013 в 14:47 -0500, Serge Hallyn пишет:
> > Quoting Peter Volkov (pva at gentoo.org):
> > > I'm using 1.0.0.alpha1 although I've tried with 0.8.0 also and I'm
> > > unable to start container with the following error:
> > > 
> > > lxc-start: Invalid argument - pivot_root syscall failed
> > > lxc-start: failed to setup pivot root
> > > lxc-start: failed to set rootfs for 'repos'
> > > lxc-start: failed to setup the container
> > > lxc-start: invalid sequence number 1. expected 2
> > > lxc-start: failed to spawn 'repos'
> > > 
> > > I've tried mount --make-private on all mount point I've thought of with
> > > no luck.
> > > Also I've tried lxc.autodev = 1 also no luck and I guess this is
> > > relevant with systemd while this systems uses openrc as init system.
> > > 
> > > Container's conf file:
> > > 
> > > lxc.arch = amd64
> > > lxc.utsname = repos
> > > lxc.rootfs = /virt/lxc/repos
> > > 
> > > Distribution Gentoo. Same config works fine on another gentoo system.
> > > Although systems are completely different I think important differences
> > > are:
> > > 1. init system: on laptop I'm using systemd while on server openrc
> > > 2. on server I have full system inside ram (system resides inside
> > > initramfs and after boot root stays in RAM on rootfs)
> > 
> > I think that's the problem.  I could be wrong, but I think it's
> > refusing ecause your root doesn't have a parent, i.e. isn't
> > mounted somewhere.
> > 
> > I suspect we want detect_shared_rootfs() updated to check for
> > your rootfs being mount #1, and also return 1 in that case
> > (meaning we will set up an environment in which you can in
> > fact pivot_root).  
> > 
> > Is such a patch something you could write and test?
> 
> Well, it's not that easy, unfortunately. For tests I just modified
> detect_shared_rootfs to return 1, so it'll detect that / is shared.
> Tried and lxc-start failed with:
> 
> lxc-start: Invalid argument - failed to mount /usr/local/lib/lxc/rootfs
> bind
> lxc-start: Failed to chroot into slave /
> 
> and really, if I try manually mount it fails:
> # mount -o bind /usr/local/lib/lxc/rootfs /usr/local/lib/lxc/rootfs
> mount: wrong fs type, bad option, bad superblock
> on /usr/local/lib64/lxc/rootfs,
>        missing codepage or helper program, or other error
>        In some cases useful info is found in syslog - try
>        dmesg | tail or so
> 
> Ok, not a problem, I've added followint in config:
> lxc.rootfs.mount = /virt/lxc/pivot_root
> /virt/lxc/pivot_root are not on rootfs partion so mount -o bind works as
> it should:
> # mount -o bind /virt/lxc/pivot_root /virt/lxc/pivot_root
> # 
> 
> Tried again and now lxc-start fails:
>  # lxc-start -f repos.conf -n repos -l DEBUG -o lxc-start-debug 
> lxc-start: Invalid argument - Failed to rbind mount /
> to /virt/lxc/pivot_root/root
> lxc-start: Failed to chroot into slave /
> lxc-start: failed to setup rootfs for 'repos'
> lxc-start: failed to setup the container
> lxc-start: invalid sequence number 1. expected 2
> lxc-start: failed to spawn 'repos'
> 
> Why does it needs to mount /? I've read the comments before
> chroot_into_slave() but I don't understand why we need to do all of
> that. Was there any discussion? 

Well finally I found problem. In
Documentation/filesystems/ramfs-rootfs-initramfs.txt it is written:

Rootfs is a special instance of ramfs (or tmpfs, if that's enabled),
which is   
always present in 2.6 systems.  You can't unmount rootfs for
approximately the  
same reason you can't kill the init process; rather than having special
code    
to check for and handle an empty list, it's smaller and simpler for the
kernel  
to just make sure certain lists can't become empty. 

So mount --bind is not supposed to work on rootfs. Then I've wrote small
init script that creates real 'tmpfs' (as opposed to 'rootfs'), moves
system root there and then switch_root there. Now lxc-start works!
Cool! :)

--
Peter.