[lxc-users] zfs disk usage for published lxd images

Tomasz Chmielewski tch at virtall.com
Mon May 16 11:46:19 UTC 2016

I've been using btrfs quite a lot and it's great technology. There are 
some shortcomings though:

1) compression only really works with compress-force mount argument

On a system which only stores text logs (receiving remote rsyslog logs), 
I was gaining around 10% with compress=zlib mount argument - not that 
great for text files/logs. With compress-force=zlib, I'm getting over 
85% compress ratio (i.e. using just 165 GB of disk space to store 1.2 TB 
data). Maybe that's the consequence of receiving log streams, not sure 
(but, compress-force fixed bad compression ratio).

2) the first kernel where I'm not getting out-of-space issues is 4.6 
(which was released yesterday). If you're using a distribution kernel, 
you will probably be seeing out-of-space issues. Quite likely scenario 
to hit out-of-space with a kernel lower than 4.6 is to use a database 
(postgresql, mongo etc.) and to snapshot the volume. Ubuntu users can 
download kernel packages from 

3) had some really bad experiences with btrfs quotas stability in older 
kernels, and judging by the amount of changes in this area on 
linux-btrfs mailing list, I'd rather wait a few stable kernels than use 
it again

4) if you use databases - you should chattr +C database dir, otherwise, 
performance will suffer. Please remember that chattr +C does not have 
any effect on existing files, so you might need to stop your database, 
copy the files out, chattr +C the database dir, copy the files in

Other than that - works fine, snapshots are very useful.

It's hard to me to say what's "more stable" on Linux (btrfs or zfs); my 
bets would be btrfs getting more attention in the coming year, as it's 
getting its remaining bugs fixed.

Tomasz Chmielewski

On 2016-05-16 20:20, Ron Kelley wrote:
> I tried ZFS on various linux/FreeBSD builds in the past and the
> performance was aweful.  It simply required too much RAM to perform
> properly.  This is why I went the BTRFS route.
> Maybe I should look at ZFS again on Ubuntu 16.04...
> On 5/16/2016 6:59 AM, Fajar A. Nugraha wrote:
>> On Mon, May 16, 2016 at 5:38 PM, Ron Kelley <rkelleyrtp at gmail.com> 
>> wrote:
>>> For what's worth, I use BTRFS, and it works great.
>> Btrfs also works in nested lxd, so if that's your primary use I highly
>> recommend btrfs.
>> Of course, you could also keep using zfs-backed containers, but
>> manually assign a zvol-formatted-as-btrfs for first-level-container's
>> /var/lib/lxd.
>>>  Container copies are almost instant.  I can use compression with 
>>> minimal overhead,
>> zfs and btrfs are almost identical in that aspect (snapshot/clone, and
>> lz4 vs lzop in compression time and ratio). However, lz4 (used in zfs)
>> is MUCH faster at decompression compared to lzop (used in btrfs),
>> while lzop uses less memory.
>>> use quotas to limit container disk space,
>> zfs does that too
>>> and can schedule a deduplication task via cron to save even more 
>>> space.
>> That is, indeed, only available in btrfs
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

More information about the lxc-users mailing list