<html><body><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; color: #333333"><div>I've figured out how to remove the subvolumes:<br></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"><strong>root@R510-LXD4-DMZ:~#</strong> btrfs subvolume list /storage</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 257 gen 83 top level 5 path lxd/common/lxd/storage-pools/Pool-1</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 258 gen 165 top level 257 path lxd/common/lxd/storage-pools/Pool-1/containers</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 259 gen 81 top level 257 path lxd/common/lxd/storage-pools/Pool-1/snapshots</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 260 gen 165 top level 257 path lxd/common/lxd/storage-pools/Pool-1/images</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 261 gen 83 top level 257 path lxd/common/lxd/storage-pools/Pool-1/custom</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 264 gen 165 top level 260 path lxd/common/lxd/storage-pools/Pool-1/images/ca0613ce3f58111feb09a4775216fad94046360a00586738f8f80c2fb20b5bc7</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 265 gen 4527 top level 258 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 497 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/3398c42dda6ee295d52d593a0e5334dd51a172826e5881ec9dff84bc37315c8d</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 498 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/b9677d2c91cd0ccc00f8b557b56c103390c777b3e2574aee3a35595224665d75</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 499 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/b62a1c869c9e9b7dbc8693b7b97c14cb7ccf09b3bf52e0d6a7656cc3bb50c312</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 500 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/5a7b74cc5a0e14d161d5ea942001b83812021260722e64da27cb315a52d6fa70</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 501 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/37b4ed38bb85b4140ab416ff74317968e4e18fd2ffadb74236cfb4b4254f785b</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 502 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/76ecc26b9fd080267c4e937f80b0993ce0e637847240f36dd3c434eae01fbb37</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 503 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/44474f3511f76cda0cb0f4227c59981e8a830fc90f17a2e9376e0f6b35842590</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 505 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/e3db5dcb5a3e25caae66eed98976d46ec46f17a0a3e3e94e0985635fc614ec54</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 508 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/6b359e6c6a5714c0e227a444601c7d4c4ab281079ebecf712b772c553e52f65f</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 511 gen 4285 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/77e3c37a5af37ff8e66fe58349afac618882ac684de3231ffe12a163e9c7211f</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 513 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/97c0031787223910a30eaf207b51f49bf29636b78c91b4ff863ade3b2712ebf2</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 514 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/5a5c9a877cb45167eafadc4d495d2d6f426fbcc3ca621ca8486ab89380aa7abb</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 515 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/6da41daadc16030e047af3a9a568266a29ee6ea109df16361b998f59c7da0d4f</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 516 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/2dea1f6a2751b66441e632e60e6b5ca56a4cdb02bcf85e1d7685ccd30e10d266</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 517 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/ffc00c903c2df58df0c7fad14dfe15436410170647da8117d0f098ab9897afc5</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 518 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/0a59b1c82fe9afa60af6ec774f1563b85cc8badd4020422c6985eed6df02f4a3</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 519 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/05e65121f1be1ff23ed768f287a2c98520c48bc245c46f1f334be0e8ee8950a4</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ID 520 gen 4508 top level 265 path lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/e72e4f68d906b2cd8713241dc752f7111133f79dae88939d8248946234ce180d</span></em><br></div><div><br data-mce-bogus="1"></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"><em><strong>root@R510-LXD4-DMZ:~#</strong> btrfs subvolume delete /storage/lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/3398c42dda6ee295d52d593a0e5334dd51a172826e5881ec9dff84bc37315c8d</em></span><br><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"><em>Delete subvolume (no-commit): '/storage/lxd/common/lxd/storage-pools/Pool-1/containers/nextcloud/rootfs/var/lib/docker/btrfs/subvolumes/3398c42dda6ee295d52d593a0e5334dd51a172826e5881ec9dff84bc37315c8d'</em></span><br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div>I still haven't figured out how to properly run docker in LXD/LXC. I've <a href="https://stackoverflow.com/posts/25885682/revisions" data-mce-href="https://stackoverflow.com/posts/25885682/revisions">seen this</a> but I'm not sure how to properly/securely do this with LXD.<br data-mce-bogus="1"></div><div><br data-mce-bogus="1"></div><div data-marker="__SIG_PRE__"><div>Thanks! <br></div><div>Lai<br></div></div><div><br></div><hr id="zwchr" data-marker="__DIVIDER__"><div data-marker="__HEADERS__"><b>From: </b>"Lai Wei-Hwa" <whlai@robco.com><br><b>To: </b>"lxc-users" <lxc-users@lists.linuxcontainers.org><br><b>Sent: </b>Thursday, December 7, 2017 4:35:41 PM<br><b>Subject: </b>[lxc-users] Problem with a docker pull & deleting btrfs subvolumes in a container<br></div><div><br></div><div data-marker="__QUOTED_TEXT__"><div id="zimbraEditorContainer" style="font-family: arial, helvetica, sans-serif; font-size: 10pt; color: rgb(51, 51, 51);" class="14" data-mce-style="font-family: arial, helvetica, sans-serif; font-size: 10pt; color: #333333;"><div>Howdy all, <br></div><br><div>Host: Xenial 16.04<br></div><div>LXD: Snappy 2.20<br></div><br><div>When I try to pull and extract code for a docker container, I get:<br></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">root@nextcloud:~# sudo docker pull collabora/code</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Using default tag: latest</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">latest: Pulling from collabora/code</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">bd97b43c27e3: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">6960dc1aba18: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">2b61829b0db5: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">1f88dc826b14: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">73b3859b1e43: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">0f0e2d01915e: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">2458b914d686: Pull complete </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">a3e2abb56fa4: Extracting [==================================================>] 972.1 MB/972.1 MB</span></em><br></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">failed to register layer: ApplyLayer exit status 1 stdout: stderr: operation not permitted</span></em><br></div><div style=""><br></div><div>Additionally, when I went to delete the btrfs subvolumes created by docker (maybe related to <a href="https://lists.linuxcontainers.org/pipermail/lxc-users/2015-February/008494.html" target="_blank">this issue</a>?):<br></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">root@nextcloud:/var/lib/docker/btrfs/subvolumes# rm -rf *</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '3398c42dda6ee295d52d593a0e5334dd51a172826e5881ec9dff84bc37315c8d': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '37b4ed38bb85b4140ab416ff74317968e4e18fd2ffadb74236cfb4b4254f785b': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '44474f3511f76cda0cb0f4227c59981e8a830fc90f17a2e9376e0f6b35842590': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '5a7b74cc5a0e14d161d5ea942001b83812021260722e64da27cb315a52d6fa70': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '6b359e6c6a5714c0e227a444601c7d4c4ab281079ebecf712b772c553e52f65f': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '76ecc26b9fd080267c4e937f80b0993ce0e637847240f36dd3c434eae01fbb37': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove '77e3c37a5af37ff8e66fe58349afac618882ac684de3231ffe12a163e9c7211f': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove 'b62a1c869c9e9b7dbc8693b7b97c14cb7ccf09b3bf52e0d6a7656cc3bb50c312': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove 'b9677d2c91cd0ccc00f8b557b56c103390c777b3e2574aee3a35595224665d75': Operation not permitted</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">rm: cannot remove 'e3db5dcb5a3e25caae66eed98976d46ec46f17a0a3e3e94e0985635fc614ec54': Operation not permitted</span></em><br></div><br><div>Which I'm assuming is the same issue causing:<br></div><div style="padding-left: 30px;" data-mce-style="padding-left: 30px;"><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">r<em>oot@nextcloud:/var/lib/docker/btrfs/subvolumes# sudo apt remove --purge docker.io </em></span><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Reading package lists... Done</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Building dependency tree </span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Reading state information... Done</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">The following packages were automatically installed and are no longer required:</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> at-spi2-core bridge-utils cgroupfs-mount containerd gconf-service gconf-service-backend gconf2 gconf2-common libatk-bridge2.0-0 libatk-wrapper-java libatk-wrapper-java-jni libatspi2.0-0 libavahi-glib1</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> libbonobo2-0 libbonobo2-common libcanberra0 libfontenc1 libgconf-2-4 libgnome-2-0 libgnome2-common libgnomevfs2-0 libgnomevfs2-common libice6 liborbit-2-0 libsm6 libtdb1 libvorbisfile3 libxaw7</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> libxcb-shape0 libxft2 libxmu6 libxt6 libxv1 libxxf86dga1 runc sound-theme-freedesktop ubuntu-fan x11-utils</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Use 'sudo apt autoremove' to remove them.</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">The following packages will be REMOVED:</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> docker.io*</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">0 upgraded, 0 newly installed, 1 to remove and 11 not upgraded.</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">After this operation, 62.7 MB disk space will be freed.</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Do you want to continue? [Y/n] y</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">(Reading database ... 34706 files and directories currently installed.)</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Removing docker.io (1.13.1-0ubuntu1~16.04.2) ...</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">'/usr/share/docker.io/contrib/nuke-graph-directory.sh' -> '/var/lib/docker/nuke-graph-directory.sh'</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Purging configuration files for docker.io (1.13.1-0ubuntu1~16.04.2) ...</span></em><br><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Nuking /var/lib/docker ...</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> (if this is wrong, press Ctrl+C NOW!)</span></em><br><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">+ sleep 10</span></em><br><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">+ umount -f /var/lib/docker/btrfs</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">umount: /var/lib/docker/btrfs: block devices are not permitted on filesystem</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">dpkg: error processing package docker.io (--purge):</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> subprocess installed post-removal script returned error exit status 32</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Processing triggers for man-db (2.7.5-1) ...</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">Errors were encountered while processing:</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;"> docker.io</span></em><br><em><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">E: Sub-process /usr/bin/dpkg returned an error code (1)</span></em><br></div><div style=""><br></div><div style="">Question 1:<br></div><div style="">How can I get past the docker pull issue?<br></div><div style=""><br></div><div style="">Question 2:<br></div><div style="">Does the btrfs subvolume issue mean that any subvolume in a container can't be removed? If it can, what's the proper way to remove them?<br></div><br><div><div>Best Regards,</div><br><div style="text-align: left;" data-mce-style="text-align: left;"><img style="" data-mce-src="https://www.robco.com/images/logo.jpg" src="https://www.robco.com/images/logo.jpg" align="left"><strong>Lai Wei-Hwa</strong><br> <span style="font-size: 10pt;" data-mce-style="font-size: 10pt;">IT Administrator</span><br> T. (514) 367-2252 ext 6308 C. (514) 218-7400 <a href="mailto:whlai@robco.com" target="_blank">whlai@robco.com</a><br> Montreal<strong><span style="color: rgb(255, 0, 0);" data-mce-style="color: #ff0000;"> -</span></strong> Toronto <strong><span style="color: rgb(255, 0, 0);" data-mce-style="color: #ff0000;">-</span></strong> Edmonton <a href="https://www.robco.com" target="_blank">www.robco.com</a><br><hr></div><div style="text-align: right;" data-mce-style="text-align: right;"><span style="color: rgb(128, 128, 128);" data-mce-style="color: #808080;">ISO 9001 / 14001</span></div><div style="text-align: left;" data-mce-style="text-align: left;"><br></div></div></div><br>_______________________________________________<br>lxc-users mailing list<br>lxc-users@lists.linuxcontainers.org<br>http://lists.linuxcontainers.org/listinfo/lxc-users<br></div></div></body></html>