[Lxc-users] Rootfs as rw overlay on top of ro directory

[Ivan Vilata i Balaguer]

Hi, sorry for being so late with the reply, I wanted to test some of
your suggestions before.

Serge Hallyn (2013-06-11 17:11:47 +0200) wrote:

> This is really cool, first off.  I'd like to extend the built-in
> overlayfs backing store support a bit, but there will also be
> room for more powerful setups using mount hooks.  I'd like to
> add your mount hook into the set of listed examples, if you don't
> mind.

Of course I don't, you may consider them in the public domain.  But
please keep on reading for more elaborated scripts. :)

> The pre-start and post-stop routines run in the monitor's namespace.
> The pre-mount routine runs in the container's namespace.  You might
> try moving the pre-mount stuff all into pre-start instead, and
> umounting them (-l) in the post-stop.  My guess is that the aufs is
> not being umounted for one reason or another.

I moved everything to pre-start and post-stop and the result is the
same, i.e. ``overlay.ext4`` is badly unmounted (needs journal recovery)
and the overlay loop device cannot be detached.  It looks like something
doesn't get cleaned up and may be holding the overlay file in use, but I
don't know where and how to look for it.  I tried reenabling the
``umountfs`` and ``umountroot`` init scripts in the container (Debian)
just in case but nothing changed.

> You also might try overlayfs, which might correctly get auto-umounted.

I managed to perform my tests on an Ubuntu Raring Ringtail i686 under
VirtualBox but the problems persist (even worse, the SquashFS loop
device also becomes undetachable).

The scripts I used this time are included below.

For ``lxc.hook.pre-mount`` version:

    #!/bin/sh
    LXC_DIR=$(dirname "$LXC_ROOTFS_PATH")

    ACTION="$3"
    case "$ACTION" in
      (pre-mount)
        mount -nt squashfs -o ro "$LXC_DIR/template.squashfs"
        "$LXC_DIR/template"
        mount -nt ext4 "$LXC_DIR/overlay.ext4" "$LXC_DIR/overlay"
        mount -nt overlayfs -o "lowerdir=$LXC_DIR/template,upperdir=$LXC_DIR/overlay" sliver "$LXC_ROOTFS_PATH" \
        || mount -nt aufs -o "br=$LXC_DIR/overlay:$LXC_DIR/template" sliver
        "$LXC_ROOTFS_PATH"
        ;;
      (*)
        echo "Invalid action: $ACTION" > /dev/null
        exit 1
        ;;
    esac

For ``lxc.hook.pre-start`` and ``lxc.hook.post-stop`` version:

    #!/bin/sh
    LXC_DIR=$(dirname "$LXC_ROOTFS_PATH")

    ACTION="$3"
    case "$ACTION" in
      (pre-start)
        mount -t squashfs -o ro "$LXC_DIR/template.squashfs"
        "$LXC_DIR/template"
        mount -t ext4 "$LXC_DIR/overlay.ext4" "$LXC_DIR/overlay"
        mount -t overlayfs -o "lowerdir=$LXC_DIR/template,upperdir=$LXC_DIR/overlay" sliver "$LXC_ROOTFS_PATH" \
        || mount -t aufs -o "br=$LXC_DIR/overlay:$LXC_DIR/template" sliver
        "$LXC_ROOTFS_PATH"
        ;;
      (post-stop)
        umount -l "$LXC_ROOTFS_PATH"
        umount -l "$LXC_DIR/overlay"
        umount -l "$LXC_DIR/template"
        ;;
      (*)
        echo "Invalid action: $ACTION" > /dev/null
        exit 1
        ;;
    esac

Thanks for your help!  Any suggestion on how to further investigate the
issue is very welcome.

Cheers,

-- 
Ivan Vilata i Balaguer -- https://elvil.net/