[Lxc-users] Container start unmounts shared bind mounts

[Serge Hallyn]

Quoting Ivan Vilata i Balaguer (ivan at selidor.net):
> Serge Hallyn (2012-02-09 19:30:29 +0100) wrote:
> 
> > Quoting Ivan Vilata i Balaguer (ivan at selidor.net):
> >> Hi all.  I'm running Debian's LXC 0.7.5 under Linux 3.2.0.  I've set up
> >> a shared mountpoint to dynamically export some host directories into one
> >> container, like this::
> >> 
> >>   # mkdir -p /lxc-shared
> >>   # mount --bind /lxc-shared /lxc-shared
> >>   # mount --make-unbindable /lxc-shared
> >>   # mount --make-shared /lxc-shared
> >
> > (I should think more before answering, but ...)
> >
> > What if you do 'mount --make-rslave /lxc-shared' here?  That should
> > prevent the container's mount actions from being forwarded to the
> > host.
> 
> Thanks for the suggestion!  That does prevent a starting container from
> unmounting bind mounts under /lxc-shared in the host, *however* it also
> renders (un)mounts performed after the --make-rslave invisible to any
> container which had access to the directory.  E.g. imagine myvm has a

Right, this was a quick test.  What you actually want to do is leave the
mount shared on the host, and have the container startup turn it into a
slave mount.  I'm not sure offhand what would be the best time to do this,
but one thing you could do is use a wrapper around lxc-start like:

mv /usr/bin/lxc-start /usr/bin/lxc-start.real

cat > /usr/bin/lxc-start.mid << EOF
mount --make-unbindable /lxc-shared
mount --make-shared /lxc-shared
exec /usr/bin/lxc-start.real $*
EOF

cat > /usr/bin/lxc-start << EOF
lxc-unshare -s MOUNT -- /usr/bin/lxc-start.mid $*
EOF

chmod ugo+x /usr/bin/lxc-start{,.mid}

You can probably do this through /var/lib/lxc/<container>/fstab entries,
but it would take some tweaking.  We could also add support for this
in the lxc config files.  I think it's a common enough request that it'd
be worth doing.

> /shared directory and this config line::
> 
>   lxc.mount.entry = /lxc-shared/myvm/ /var/lib/lxc/debtest/rootfs/shared/ none defaults,bind 0 0
> 
> Then::
> 
>   host# mkdir -p /lxc-shared
>   host# mount --bind /lxc-shared /lxc-shared
>   host# mount --make-shared /lxc-shared
>   host# lxc-start -n myvm -d
>   # myvm sees /lxc-shared/myvm at /shared
>   host# mkdir -p /lxc-shared/myvm/foo
>   host# mount --bind /tmp /lxc-shared/myvm/foo
>   # myvm sees mounted /shared/foo
>   host# mount --make-rslave /lxc-shared
>   # myvm still sees mounted /shared/foo
>   host# lxc-start -n myothervm -d
>   # myvm still sees mounted /shared/foo
>   host# mkdir -p /lxc-shared/myvm/bar
>   host# mount --bind /tmp /lxc-shared/myvm/bar
>   # myvm sees /shared/bar but nothing mounted on it!
> 
> A workaround I found is bind mounting the desired directory *in the
> container* (which requires not dropping the sys_admin capability)::
> 
>   host# mkdir -p /lxc-shared
>   host# mount --bind /lxc-shared /lxc-shared
>   host# mount --make-shared /lxc-shared
>   host# lxc-start -n myvm -d
>   # myvm sees /lxc-shared/myvm at /shared
>   host# mkdir -p /lxc-shared/myvm/foo
>   host# mount --bind /tmp /lxc-shared/myvm/foo
>   # myvm sees mounted /shared/foo
>   myvm# mount --bind /shared/foo /mnt/foo
>   host# lxc-start -n myothervm -d
>   # host's /lxc-shared/myvm/foo gets unmounted
>   # myvm sees /shared/foo but nothing mounted on it
>   # myvm still sees mounted /mnt/foo
>   host# mkdir -p /lxc-shared/myvm/bar
>   host# mount --bind /tmp /lxc-shared/myvm/bar
>   # myvm sees mounted /shared/bar
>   myvm# mount --bind /shared/bar /mnt/bar
>   # and so on...
> 
> However, the question still remains: *Why on Earth does starting a
> container unmount all bind mounts under a shared mount???*
> 
> Doesn't it look like a bug to you?

No, when a container starts up, it mounts its new root under, say,
/usr/lib/lxc/, and mounts other directories under there.  Then it
does pivot_root (see man 8 pivot_root), so now /usr/lib/lxc is its
'/', and the old '/' and all its submounts are now mounted on '/old'.
Then the container startup recursively unmounts /old, including
/old/lxc-shared.

That umount of /old/lxc-shared is what is getting propagated to
the host mount.

-serge

> Thanks & cheers!
> 
> >> Now I bind mount the host directory under the shared directory::
> >> 
> >>   # mkdir -p /lxc-shared/myvm/foo
> >>   # mount --bind /tmp /lxc-shared/myvm/foo
> >> The problem is that whenever I start any container, /lxc-shared/myvm/foo
> >> gets unmounted (even if it has processes working under it!).  This
> >> affects bind mounts only if they are under shared mountpoints, e.g. if I
> >> also do this mount on the host::
> >> 
> >>   # mount --bind /tmp /mnt
> >> 
> >> It survives after starting the container.
> >> 
> >> Does anyone know why does this happen?  Should I file a bug report?
> >> Thanks a lot!
> -- 
> Ivan Vilata i Balaguer -- https://elvil.net/
> 
> 
> ------------------------------------------------------------------------------
> Virtualization & Cloud Management Using Capacity Planning
> Cloud computing makes use of virtualization - but cloud computing 
> also focuses on allowing computing to be delivered as a service.
> http://www.accelacomm.com/jaw/sfnl/114/51521223/
> _______________________________________________
> Lxc-users mailing list
> Lxc-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users