[lxc-users] About size of images
Michael H. Warfield
mhw at WittsEnd.com
Sat Aug 30 14:25:26 UTC 2014
On Sat, 2014-08-30 at 09:02 +0200, Marcel Sánchez Toledano wrote:
> Hi again,
>
> Anyone who knows about this can tell us something? I need some
> information about this.
It's very hard to say much about it when you're comparing apples to
oranges.
To start off with, you're comparing a file system image size to the disk
utilization within a file system. The two do not directly compare
unless the file system is full and even then it's a crap shoot comparing
the result from "ls -l" to that from "du". There's too many factors and
too many variables that go into it.
What kind of image is that .fs?
Is it compressed?
Is it ext[234]?
Is it squashfs? In the case of compressed file systems, such as
squashfs, the size of the file system can actually be smaller than the
effective contents, since their space utilization is based on their
uncompressed size (but that's the opposite of what you've got - I'm just
pointing it out to illustrate why the comparison is meaningless).
Is that .fs image itself a "sparse file" (i.e. not all disk space is
allocated throughout the image, areas that are blocks of 0's may not be
allocated a disk block). An "ls -l" will show you a file "size" but it
may not be the true disk usage of the file if it's sparse.
What happens when you mounted it over a loop device, enter the mount
point and do a du on it? That would, at least, come closer to a more
valid comparison. How about the results from "df" in that file system?
Is that file system image intended to contain space for home directories
or var space? If it's intended for heavy weight virtualization, it
probably has considerable empty space for home directories (I usually
allocation 8G-16G for my heavy weight VM images).
The fact that both the images you show are 2.0G implies, to me, that
they are probably pre-allocated images which are not fully populated and
may even be "sparse files", so "ls -l" is not coming anywhere remotely
NEAR the actual size of the contents and might not even be indicative of
the disk space used in the host file system. Often, an image like that
is created as a fixed size that will contain the largest possible
installation, zeroed out (or left sparse), and a file system build in it
to that fixed size. Then you create your installation in that image and
it may only occupy 10% of the file system. But, you don't care. When
you compress it, the compression utilities are very efficient at
compressing those empty spaces and it costs you very little in your
downloadable size.
A heavy weight file system image (and this could be an LXC backing store
image as well) generally has to contain space for home and var
directories, it's not just the operating system binaries. An LXC
container, OTOH, back by a directory, is sharing the file system space
with the host OS and doesn't need separately allocated space for home
directories.
The biggest difference, though, is very likely to be installation
features. The LXC templates create minimal installations. No X, no
GUI, no desktop. Just a bare bones system which you can then flesh out
to what you want. That's by intent. That minimizes the size of the
download images (in the case of the download template), minimizes the
size of the template caches, speeds installation, and allows the freedom
to customize the images without excessive and needless redundancy of
unused packages.
For heavy weight images running under a hypervisor, these are generally
created from full blown installation images (CDs and DVDs) and contain a
full basic GUI standard installation plus options.
Compare you're installation packages and see what's installed. Run dpkg
in those file systems and compare what packages you've got installed in
there.
Installations can easily vary over a couple of orders of magnitude
depending on what you install. I can cram an install of NST (Network
Security Toolkit - a Fedora variation) onto a 650M CD, and that's WITH a
graphical environment. Puppy Linux (Slackware based) still runs in less
than 200M. Without a graphical environment, you could still cram some
slimmed down distro images, like TinyCore (which still chimes in at a
whopping 14M for the slim image!), onto a 50M BBC (Bootable Business
Card - a shaped mini-CD). There are a number of "runlive" images
available for many distributions that will run from a CD (generally from
a squashfs image). OTOH, a full installation of Debian, or Ubuntu, or
Fedora, or CentOS is going to whack out at several gig if not 10's of
gig just for the installation binaries, once their uncompressed.
Another reason you may not have gotten answers is that few of us would
have any working knowledge of SIMCTL or VNUML. I don't. I had to look
them up. And I'm pretty involved with virtualization on many levels.
There again, you're comparing apples to oranges. This would be like
comparing and LXC container directory to a VMware, VirtualBox, LKVM, or
Xen file system image (all of which I have worked with). The results
would be just as meaningless.
Even though VNUML looks to be a "user mode linux", it's still running
it's own kernel like a full hypervisor would and would be using file
system images like heavyweight virtualization would. You just can not
compare the size of a file system image to the contents of the file
system in cases like these. That's like asking why a partition is so
much larger that the files it contains (and could even be smaller if it
was a squashfs).
Honestly, I'm totally unimpressed with the differences in sizes. I just
think the comparison is meaningless without understanding all the
variables that go into it. Even understanding all the variables, the
comparison of file system image size (ls -l) to the utilization of the
file system by its contents (du) is still meaningless. Apples and
oranges. They're both fruit and that's about it.
> Thank you agin,
Regards,
Mike
>
>
> Marcel Sánchez Toledano
>
>
>
>
>
> 2014-07-17 14:46 GMT+02:00 Marcel Sánchez Toledano
> <marcelsanchezt at gmail.com>:
> You mean symbolic links? I don't think so.
>
>
> Anyway, a du -hs in that folder should return the real size,
> including the sizes of symbolic links original files.
>
>
>
>
>
> Marcel Sánchez Toledano
>
>
>
>
>
> 2014-07-17 14:19 GMT+02:00 Oliver Rath <rath at mglug.de>:
>
> Possybly your lxc installation uses links instead of
> real files?
>
> Regards
> Oliver
>
>
>
> On 17.07.2014 14:12, Marcel Sánchez Toledano wrote:
>
> > Hi,
> >
> >
> > I'm right now working in a project comparing SIMCTL
> > (VNUML utility, UML) with LxC. In VNUML the aprox.
> > size of virtual machines filesystems located
> > in /usr/share/vnuml/filesystems are 2GB:
> >
> >
> > marcel at ubuntu:/usr/share/vnuml/filesystems$ ls -lh
> >
> > total 4.1G
> > -rw-r--r-- 1 root root 2.0G Oct 9 2013 debian5.fs
> > -rw-r--r-- 1 root root 2.0G Jul 3 07:52 debian6.fs
> >
> >
> >
> >
> > And the size of a LxC rootfs is 246MB:
> >
> >
> > Imatge inserida 1
> >
> >
> >
> >
> >
> > How is this possible and which is the explanation of
> > this? The difference of sizes is huge!
> >
> >
> > Thanks,
> >
> >
> > Marcel Sánchez Toledano
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140830/db2a1cef/attachment.sig>
More information about the lxc-users
mailing list