[lxc-users] [debian/stretch64] lxc-copy: Snapshot with OverlayFS backingstorage fails with Linux-4.9.y

[Sedat Dilek]

[ Please CC me - I am not subscribed to lxc-users and debian-kernel MLs ]

Hi,

I am playing with LXC and OverlayFS.

The Debian-kernel shipped with stretch64 fails when using lxc-copy for
a snapshot.
Later I tried the latest Debian-kernel from stretch-backports
APT-repository [0] and the issue does *not* appear.

Debian/stretch64 ships LXC v2.0.7.

root# lxc-copy --version
2.0.7

[ Instructions to reproduce ]

[ Create LXC-Container ]

root# lxc-create -n stretch64-ovl -B overlayfs -t download -- -d
debian -r stretch -a amd64

[ GOOD: linux-image-4.12.0-0.bpo.2-amd64 (4.12.13-1~bpo9+1) ]

root# lxc-copy -n stretch64-ovl -N snapshot1 -B overlay -s
--logfile=lxc-copy-log_kernel-bpo9.txt --logpriority=DEBUG

[ BAD: linux-image-4.9.0-4-amd64 (4.9.51-1) ]

root at iniza:~# lxc-copy -n stretch64-ovl -N snapshot2 -B overlay -s
--logfile=lxc-copy-log_kernel-deb9.txt --logpriority=DEBUG

I found lxc-issue #1395 [1] where a similiar problem was reported

Can this be a kernel issue?

Miklos and Christian any idea?

Do you need further informations and/or have instructions on debugging this?

Thanks for any hints.

Regards,
- Sedat -

P.S.: I have attached the LXC configs of stretch64-ovl & snaptshot1
and the lxc-copy logfiles.

[0] https://backports.debian.org/Instructions/
[1] https://github.com/lxc/lxc/issues/1395
-------------- next part --------------
       lxc-copy 20171025093206.756 WARN     lxc_confile - confile.c:config_pivotdir:1910 - lxc.pivotdir is ignored.  It will soon become an error.
       lxc-copy 20171025093206.756 WARN     lxc_confile - confile.c:config_pivotdir:1910 - lxc.pivotdir is ignored.  It will soon become an error.
       lxc-copy 20171025093206.756 INFO     lxcoverlay - bdev/lxcoverlay.c:ovl_mount:436 - Overlayfs: Mounted /var/lib/lxc/stretch64-ovl/rootfs onto /var/lib/lxc/stretch64-ovl/rootfs with options upperdir=/var/lib/lxc/stretch64-ovl/delta0,lowerdir=/var/lib/lxc/stretch64-ovl/rootfs,workdir=/var/lib/lxc/stretch64-ovl/olwork.
       lxc-copy 20171025093206.757 INFO     lxcoverlay - bdev/lxcoverlay.c:ovl_mount:436 - Overlayfs: Mounted /var/lib/lxc/stretch64-ovl/rootfs onto /var/lib/lxc/snapshot2/rootfs with options upperdir=/var/lib/lxc/snapshot2/delta0,lowerdir=/var/lib/lxc/stretch64-ovl/rootfs,workdir=/var/lib/lxc/snapshot2/olwork.
       lxc-copy 20171025093207.420 ERROR    lxcoverlay - bdev/lxcoverlay.c:ovl_rsync:717 - rsyncing /var/lib/lxc/stretch64-ovl/rootfs to /var/lib/lxc/snapshot2/rootfs
       lxc-copy 20171025093207.421 ERROR    lxcoverlay - bdev/lxcoverlay.c:ovl_do_rsync:757 - copying overlayfs delta
       lxc-copy 20171025093207.421 ERROR    bdev - bdev/bdev.c:bdev_copy:388 - failed getting pathnames for cloned storage: overlayfs:/var/lib/lxc/stretch64-ovl/rootfs:/var/lib/lxc/stretch64-ovl/delta0
       lxc-copy 20171025093207.422 ERROR    lxc_container - lxccontainer.c:copy_storage:2867 - Error copying storage.
       lxc-copy 20171025093207.430 INFO     lxc_container - lxccontainer.c:container_destroy:2414 - Destroyed directory for snapshot2
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config_snapshot1
Type: application/octet-stream
Size: 865 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20171025/cbca1812/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config_stretch64-ovl
Type: application/octet-stream
Size: 878 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20171025/cbca1812/attachment-0001.obj>
-------------- next part --------------
       lxc-copy 20171025091007.914 WARN     lxc_confile - confile.c:config_pivotdir:1910 - lxc.pivotdir is ignored.  It will soon become an error.
       lxc-copy 20171025091007.914 WARN     lxc_confile - confile.c:config_pivotdir:1910 - lxc.pivotdir is ignored.  It will soon become an error.
       lxc-copy 20171025091007.914 INFO     lxcoverlay - bdev/lxcoverlay.c:ovl_mount:436 - Overlayfs: Mounted /var/lib/lxc/stretch64-ovl/rootfs onto /var/lib/lxc/stretch64-ovl/rootfs with options upperdir=/var/lib/lxc/stretch64-ovl/delta0,lowerdir=/var/lib/lxc/stretch64-ovl/rootfs,workdir=/var/lib/lxc/stretch64-ovl/olwork.
       lxc-copy 20171025091007.914 INFO     lxcoverlay - bdev/lxcoverlay.c:ovl_mount:436 - Overlayfs: Mounted /var/lib/lxc/stretch64-ovl/rootfs onto /var/lib/lxc/snapshot1/rootfs with options upperdir=/var/lib/lxc/snapshot1/delta0,lowerdir=/var/lib/lxc/stretch64-ovl/rootfs,workdir=/var/lib/lxc/snapshot1/olwork.
       lxc-copy 20171025091008.114 INFO     lxc_container - lxccontainer.c:copy_file:2641 - Error stat'ing /var/lib/lxc/stretch64-ovl/lxc_rdepends
       lxc-copy 20171025091008.114 INFO     lxc_container - lxccontainer.c:copy_rdepends:2800 - Error copying reverse dependencies
       lxc-copy 20171025091008.136 INFO     lxcoverlay - bdev/lxcoverlay.c:ovl_mount:436 - Overlayfs: Mounted /var/lib/lxc/stretch64-ovl/rootfs onto /usr/lib/x86_64-linux-gnu/lxc/rootfs with options upperdir=/var/lib/lxc/snapshot1/delta0,lowerdir=/var/lib/lxc/stretch64-ovl/rootfs,workdir=/var/lib/lxc/snapshot1/olwork.
       lxc-copy 20171025091008.136 INFO     lxc_conf - conf.c:run_script_argv:424 - Executing script "/usr/share/lxc/hooks/clonehostname" for container "snapshot1", config section "lxc".
       lxc-copy 20171025091008.166 INFO     lxc_copy_ui - tools/lxc_copy.c:do_clone:376 - Created snapshot1 as snapshot of stretch64-ovl