[lxc-users] Strange freezes with btrfs backend

[Fajar A. Nugraha]

On Sat, Dec 3, 2016 at 7:56 PM, Ron Kelley <rkelleyrtp at gmail.com> wrote:

> My 0.02
>
> We have been using btrfs in production for more than a year on other
> projects and about 6mos with LXD.  It has been rock solid.  I have multiple
> LXD servers each with >20 containers. We have a separate btrfs filesystem
> (with compression enabled) to store the LXD containers. I take nightly
> snapshots for all containers, and each server probably has 2000 snapshots.
> The only issue thus far is the IO hit when deleting lots of snapshots at
> one time.  You need to delete a few (10 at a time), pause for 60secs, then
> delete the next 10.
>

Ultimately, IMHO it comes down to what you're comfortable with best.

I like the fact that btrfs can be used in nested lxd, but I didn't like the
fact that you can't get "disk usage of one container" in btrfs. My
compromise so far was to always use zfs, but assign btrfs-formatted zvol
when I need nested lxd.



> I have used ZFS in Linux in the past and could never get adequate
> performance - regardless of tuning or amount of RAM given to ZFS.  In fact,
> I started using ZFS for our backup server (64TB raw storage with 32GB RAM)
> but had to move back to XFS due to severe performance issues.  Nothing
> fancy; I did a by-the-bok install and enabled compression and snapshots. I
> tried every tuning option available (including SSD for L2-ARC). Nothing
> would improve the performance.
>

AFAIK the recommendation is 1GB RAM (for zfs use) for every 1TB of raw
disk. That is on top of whatever amount of RAM required by the OS/app.
Depending on your load, SLOG might be more useful than L2ARC (in fact, when
configured incorrectly, L2ARC can do more harm than good). Testing this is
easy enough though: if you experience much better performance with
"sync=disabled", then you need SLOG.


> To the OP: are you sure btrfs is causing your issues?  Have you traced the
> OP activity during the hiccup moments?
>
> ... hence my earlier recommendation: htop, check syslog for OOM messages.

@Pierce: Add "iostat -mx 3" to that (especially to monitor IOPS usage), and
also Tomasz's advice: don't use a disk image file.
if your provider doesn't allow additional disk images (or makes it REALLY
hard to do so, like many cheap KVM-SSD VPS provider), then I highly
recommend you check out EC2: their free tier includes vps with 1GB RAM, and
you can easily add additional block devices.

-- 
Fajar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20161204/0c12073d/attachment.html>