[Lxc-users] Can't start containers

[Serge Hallyn]

Quoting Milan Zamazal (pdm at zamazal.org):
> I've got the same problem on current Debian wheezy.  The containers
> start fine on boot, but it's impossible to (re)start any container
> later:
> 
>   # lxc-start -n test
>   lxc-start: Invalid argument - pivot_root syscall failed
>   lxc-start: failed to setup pivot root
>   lxc-start: failed to set rootfs for 'test'
>   lxc-start: failed to setup the container
>   lxc-start: invalid sequence number 1. expected 2
>   lxc-start: failed to spawn 'test'
> 
> The failure happens in the call
> 
>   pivot_root(".", path)
> 
> in conf.c:setup_rootfs_pivot_root, with the error message "Invalid
> argument".  `path' value is "/mnt/lxc/mnt", current directory is
> "/mnt/lxc".  This seems to be correct, or at least intended.  The
> directories in /mnt/lxc are mounted as follows:
> 
>   /dev/mapper/raid10-root on /mnt/lxc type ext4 (rw,relatime,errors=remount-ro,user_xattr,acl,barrier=1,stripe=384,data=ordered)
>   none on /mnt/lxc/proc type proc (rw,relatime)
>   none on /mnt/lxc/dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
>   none on /mnt/lxc/tmp type tmpfs (rw,relatime,size=16384k)
>   udev on /mnt/lxc/dev/snd type devtmpfs (rw,relatime,size=1861084k,nr_inodes=465271,mode=755)
>   devpts on /mnt/lxc/dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
>   devpts on /mnt/lxc/dev/tty1 type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
>   devpts on /mnt/lxc/dev/tty2 type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> 
> (/dev/mapper/raid10-root is the root file system of the host.)
> 
> My container configuration entries are:
> 
>   lxc.rootfs = /var/lib/vservers/test
>   lxc.rootfs.mount = /mnt/lxc
> 
> And the container fstab:
> 
>   none	/var/lib/vservers/test/proc		proc	defaults		0 0
>   none	/var/lib/vservers/test/dev/pts		devpts	gid=5,mode=620		0 0
>   none	/var/lib/vservers/test/tmp		tmpfs	size=16m,mode=1777	0 0
>   /dev/snd	/var/lib/vservers/test/dev/snd	none	bind			0 0
> 
> I tried to test pivot_root from command line after an unsuccessful
> container start:
> 
>   # cd /mnt/lxc
>   # ls
>   bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root run  sbin  selinux  srv  sys  tmp  usr  var
>   # ls mnt
>   # pivot_root . /mnt/lxc/mnt
>   pivot_root: Invalid argument
> 
> I've no idea whether the pivot_root call should succeed or not under
> these circumstances and why it succeeds on boot but doesn't succeed
> later when lxc-start is invoked from a command line. 
> 
> Does anybody know what can be wrong with the pivot_root call?

Well, pivot_root can fail due to various and sundry MS_SHARED settings on
/mnt/lxc, /mnt/lxc/mnt, and whatever /mnt/lxc was mounted on top of.
What is odd though is that your first call succeeds and the rest fail.
Does /proc/self/mounts on the host change after you successfully start
the container the first time?

Can you do 'lxc-start -n test -l DEBUG -o test.debug', twice, and send
us the resulting test.debug file?

-serge