[lxc-users] Strange freezes with btrfs backend

[Ron Kelley]

My 0.02

We have been using btrfs in production for more than a year on other 
projects and about 6mos with LXD.  It has been rock solid.  I have multiple 
LXD servers each with >20 containers. We have a separate btrfs filesystem 
(with compression enabled) to store the LXD containers. I take nightly 
snapshots for all containers, and each server probably has 2000 snapshots. 
The only issue thus far is the IO hit when deleting lots of snapshots at 
one time.  You need to delete a few (10 at a time), pause for 60secs, then 
delete the next 10.

I have used ZFS in Linux in the past and could never get adequate 
performance - regardless of tuning or amount of RAM given to ZFS.  In fact, 
I started using ZFS for our backup server (64TB raw storage with 32GB RAM) 
but had to move back to XFS due to severe performance issues.  Nothing 
fancy; I did a by-the-bok install and enabled compression and snapshots. I 
tried every tuning option available (including SSD for L2-ARC). Nothing 
would improve the performance.

To the OP: are you sure btrfs is causing your issues?  Have you traced the 
OP activity during the hiccup moments?




On December 3, 2016 7:37:21 AM "Fajar A. Nugraha" <list at fajar.net> wrote:

> On Sat, Dec 3, 2016 at 6:01 PM, Sergiusz Pawlowicz <sergiusz at pawlowicz.name>
> wrote:
>
>> > You'd need to set arc to be as small as possible:
>> > # cat /etc/modprobe.d/zfs-arc-max.conf
>> > options zfs zfs_arc_max=67108865
>>
>> What is a sense of using ZFS if you don't use its cache? Non sense. it
>>
>
> - excellent integration with lxd
> - data integrity verification using checksum
> - thin-lvm-like space management
> - send/receive
> - compression
> - much more mature compared to btrfs
>
>
>> will work slower and less reliable than ext4.
>>
>>
> I never said it was faster.
>
> In general, zfs WILL be slower - to some extent - compared to ext4. Just
> like ext4 (presumably with LVM and raid/mirror) will be slower compared to
> writing to raw disk directly - especially if you also exclude any kind of
> raid/mirror and volume manager.
>
> To give more perspective to my particular use case, my EC2 zfsroot AMI only
> use 1GB EBS thanks to lz4 compression. And that's with around 400 MB free
> space. Thanks to zfs snapshot/clone, I can also use clones of my root as
> containers (which is more efficient compared to LVM snapshots or overlay)
>
> Is it a suitable solution for everyone? No.
> Does it work for my use case? Yes. MUCH more so compared to ext4 or btrfs.
> Will it work for Pierce's use case? I believe so.
>
> --
> Fajar
>
>
>
> ----------
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20161203/7d21c2f0/attachment.html>