[Lxc-users] Container size minialisation

[Ivan Vilata i Balaguer]

Zhu Yanhai (2011-12-13 12:00:20 +0800) wrote:

> > On 12/12/2011 09:47 AM, István Király - LaKing wrote:
> >
> > As next step I would like to minimize container size. My question is, what
> > the best, most elegant and fail proof  technique for that?
> 
> It's not only an issue about disk space, it's about page cache cost.  If the
> under level files are shared, the OS see them as unique inodes, so the OS
> allocate only one copy for each one inode. However if we copy them as
> separate directories, each of them will occupy one copy in the page cache,
> resulting in large cost of memory.  I think a better approach here is a
> file-level copy-on-write trick in filesystem or some other proper
> layers. Initially all files are shared in the host, and copy one as a
> private item at the first time it is touched in one containers.

I've carried out some tests with union mount points and Btrfs seeds and CoW
copies, and all of them seem to work without problems with LXC.

For union mount points, you can keep a base rootfs under a read-only
directory, and then mount a writable directory on top of that for each
container's rootfs, like this:

  /var/lib/lxc/.base/debian-squeeze (RO, don't change it!)
  + /var/lib/lxc/.store/lxc01/ (RW) = /var/lib/lxc/lxc01/rootfs/ (RW)
  + /var/lib/lxc/.store/lxc02/ (RW) = /var/lib/lxc/lxc02/rootfs/ (RW)

Changed files are written under ".store/lxcNN".  Debian includes AuFS for
union mount points.

A similar solution are Btrfs seeds: you have a read-only seed Btrfs and
writable images (e.g. files via loop devices) on top, like this:

  /dev/loop0 (var/lib/lxc/.base/debian-squeeze.btrfs) (RO, seed)
  + /dev/loop1 (/var/lib/lxc/.store/lxc01.raw) (RW) = /var/lib/lxc/lxc01/rootfs/ (RW)
  + /dev/loop2 (/var/lib/lxc/.store/lxc02.raw) (RW) = /var/lib/lxc/lxc02/rootfs/ (RW)

Changed blocks are stored under ".store/lxcNN.raw" (which should be big
enough).

As for CoW copies in Btrfs, you can use simple CoW copies:

  # cp -a --reflink /var/lib/lxc/.base/debian-squeeze /var/lib/lxc/lxc01/rootfs/

or you can create a subvolume out of /var/lib/lxc/.base/debian-squeeze in your
Btrfs filesystem and /var/lib/lxc/lxc01/rootfs/ as a snapshot of that
subvolume.  In both cases only changed blocks are stored on disc, but I've
found subvolumes to be a lot faster on creation and removal.

Hope that helps,
-- 
Ivan Vilata i Balaguer -- https://elvil.net/