[Lxc-users] Container size minialisation

[Zhu Yanhai]

Hi,

2011/12/13 Derek Simkowiak <derek at simkowiak.net>:
>> I'm trying to compose a system, where lxc containers behave like virtual
>> hosts for a web server.
>
>     It is possible to use shared, read-only mounts between the host and
> containers.  However, you will need to carefully consider your security
> requirements and maintenance procedures.
>
>     If you are using shared mounts, then upgrading software on one container
> will update files in the other containers.  That could be a problem, because
> software updates usually include a post-install script that is executed at
> the end of an upgrade (like a postinst file in the .deb package).  These
> scripts may restart a service, update a configuration file in /etc/, or even
> prompt the user for some input.
>
>     So what will happen when you update the LAMP files on your shared mount,
> but your LXC containers don't do a corresponding server restart, or config
> migration?  Things will probably break at some point.
>
>     Also, there are several security risks to consider.  A shared /home/
> directory would also (implicitly) share everyone's .ssh/authorized_keys
> files (which will grant OpenSSH access to all your containers).  You would
> also need to be sure that all SSL and SSH host certs are independently
> managed.  Using a single certificate for many hosts is not secure.  Apache
> and OpenSSH keep their certs under /etc/, but Tomcat does not iirc.
>
>     Also, UIDs and GIDs are shared on the filesystem, so a root user in any
> container would be able to alter any file in any other container (unless
> it's a read-only mount from an external fstab file, and the "sys_admin"
> capability is dropped in your lxc.conf).  What's worse, if you have
> different /etc/passwd or /etc/group files in the containers, then group id
> "121" might be the group "admin" in one container, but something else
> entirely in another container.  The shared filesystem only stores the
> integer group ID, not the actual group membership or resulting sudo
> permissions.
>
>     Because of these complications, I have decided to give each LXC
> container its own, full filesystem.  Unfortunately that "wastes" a few
> hundred megs of disk space for each container, because the files are mostly
> redundant in /usr/, /var/, etc.  However, disk space is very cheap, and the
> value of having a standalone container is more than worth it to me.
>
>
>
>> Has anyone any experience with this technique?
>
>     I include a sample configuration for a shared filesystem with my
> container creation script.  It is disabled by default, but you can read
> through the configuration to get an idea.  You can download it from here:
>
> http://derek.simkowiak.net/lxc-ubuntu-x/
>
>
> Thanks,
> Derek Simkowiak
>
>
> On 12/12/2011 09:47 AM, István Király - LaKing wrote:
>
> Hi folks.
>
> I'm trying to compose a system, where lxc containers behave like virtual
> hosts for a web server.
>
> As next step I would like to minimize container size. My question is, what
> the best, most elegant and fail proof  technique for that?
>
> At this moment I'm thinking of a "master container" and "slave containers"
> where the /usr folder for example in the slave containers is a mount from
> the master container. That gives a significant size drop already, from 400
> to 40 megabytes.
>
> I would like to keep the containers really minimal. 4 megabyte should be
> small enough.
>
> Lets say only some important files in /etc ....
>
> Has anyone any experience with this technique?
>
> Thank you for sharing.
>
> greetings,
> István Király
>
>
> LaKing at D250.hu
>
> D250 Laboratories
> www.D250.hu
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Lxc-users mailing list
> Lxc-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users
>
>
>
> ------------------------------------------------------------------------------
> Learn Windows Azure Live!  Tuesday, Dec 13, 2011
> Microsoft is holding a special Learn Windows Azure training event for
> developers. It will provide a great way to learn Windows Azure and what it
> provides. You can attend the event by watching it streamed LIVE online.
> Learn more at http://p.sf.net/sfu/ms-windowsazure
> _______________________________________________
> Lxc-users mailing list
> Lxc-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users
>

It's not only an issue about disk space, it's about page cache cost.
If the under level files are shared, the OS see them as unique inodes,
so the OS allocate only one copy for each one inode. However if we
copy them as separate directories, each of them will occupy one copy
in the page cache, resulting in large cost of memory.
I think a better approach here is a file-level copy-on-write trick in
filesystem or some other proper layers. Initially all files are shared
in the host, and copy one as a private item at the first time it is
touched in one containers.

--
Thanks,
Zhu Yanhai