[Lxc-users] pivot root failures when "/" is mounted as "shared"

[Michael H. Warfield]

On Mon, 2012-11-05 at 15:19 +0100, Peter Simons wrote:
> Hi Michael,

>  > If you've tested that and you've seen how it breaks, could you post
>  > the version of systemd you are running and the error messages along
>  > with some config examples here so we can see them?

> I'm running lxc-0.8.0-rc2 on a systemd-195 host. When I try to start a
> container, I get the following error:

>   # lxc-start -n ubuntu
>   lxc-start: Invalid argument - pivot_root syscall failed
>   lxc-start: failed to setup pivot root
>   lxc-start: failed to set rootfs for 'ubuntu'
>   lxc-start: failed to setup the container
>   lxc-start: invalid sequence number 1. expected 2
>   lxc-start: failed to spawn 'ubuntu'
>   lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/systemd/system/lxc/ubuntu'

> The container is an Ubuntu Precise, created by lxc-create. The lxc
> config file and fstab are attached below, at the end of the message.

Ok...  So you DO have the pivot root issue.  And that confirms for me
that 195 is going to be problematical, which will be Fedora 18, most
likely.  I've got 195 running quite nicely in a container (with Serge's
mods).

>  > A short term work-around is to simply remount the root tree to
>  > private before invoking LXC.

> That work-around comes with significant side-effects for me. Running

>  # mount --make-rprivate /
>  # lxc-start -n ubuntu

> gives me a working container, alright, but it also screws up my host
> system real bad. The most obvious problem is that my /dev/pts directory
> has disappeared, so I can no longer allocate a PTY:

Yuck.  I have my containers running on another partition.  Don't know
what impact that would have.

Sounds like there's may be some heartburn over that decision by systemd
as well since it may allow mounts in the container to leak out into the
host.

>   # xterm
>   xterm: Error 32, errno 2: No such file or directory
>   Reason: get_pty: not enough ptys

This was in the host?  That seems very strange.  Then too, it might not
be necessary to do the "recursive private" (make-rprivate) either, just
make-private on the partition which contains the containers.

> /dev/pts is not the only mount point that's gone. I've also lost the
> following other ones:
> 
>   --- /tmp/old	2012-11-05 15:17:27.205674924 +0100
>   +++ /tmp/new	2012-11-05 15:17:42.297486250 +0100
>   @@ -3,21 +3,7 @@
>    none on /sys type sysfs (rw,relatime)
>    none on /dev type devtmpfs (rw,relatime,size=206136k,nr_inodes=220097,mode=755)
>    none on /run type tmpfs (rw,relatime,size=206136k,mode=755)
>   -none on /sys/kernel/security type securityfs (rw,relatime)
>    /dev/disk/by-label/root on / type ext4 (rw,noatime,nodiratime,data=ordered)
>    none on /tmp type tmpfs (rw,relatime)
>   -tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,relatime)
>   -none on /proc/bus/usb type usbfs (rw,relatime)
>   -devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=3,mode=620,ptmxmode=000)
>    tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,mode=755)
>   -cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,release_agent=/nix/store/i1qpxmqwwn247y59lf2cpra5iw1rgvrh-systemd-195/lib/systemd/systemd-cgroups-agent,name=systemd)
>   -cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
>   -cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
>   -cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
>   -cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
>   -cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
>   -cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
>   -hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
>   -mqueue on /dev/mqueue type mqueue (rw,relatime)
>   -debugfs on /sys/kernel/debug type debugfs (rw,relatime)
>    /dev/sda1 on /boot type ext4 (rw,noatime,nodiratime,data=ordered)
> 
> That doesn't seem right?
> 
> Take care,
> Peter

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20121105/faccf95e/attachment.pgp>