[lxc-users] Block devices not permitted on file system
Christian Brauner
christianvanbrauner at gmail.com
Sun Feb 1 16:09:41 UTC 2015
On Sun, Feb 01, 2015 at 12:00:01PM +0000, lxc-users-request at lists.linuxcontainers.org wrote:
> From serge.hallyn at ubuntu.com Sun Feb 1 15:48:08 2015
> From: serge.hallyn at ubuntu.com (Serge Hallyn)
> Date: Sun, 1 Feb 2015 15:48:08 +0000
> Subject: [lxc-users] Block devices not permitted on file system
> In-Reply-To: <20150201142655.GA1478 at gmail.com>
> References: <mailman.2.1422792001.23450.lxc-users at lists.linuxcontainers.org>
> <20150201142655.GA1478 at gmail.com>
> Message-ID: <20150201154808.GB16770 at ubuntumail>
>
> Quoting Christian Brauner (christianvanbrauner at gmail.com):
> > On Sun, Feb 01, 2015 at 12:00:01PM +0000, lxc-users-request at lists.linuxcontainers.org wrote:
> > > Date: Sun, 1 Feb 2015 07:38:57 +0000
> > > From: Serge Hallyn <serge.hallyn at ubuntu.com>
> > > To: LXC users mailing-list <lxc-users at lists.linuxcontainers.org>
> > > Subject: Re: [lxc-users] Block devices not permitted on file system
> > >
> > > Quoting Christian Brauner (subroutinecall at gmail.com):
> > > > Hello,
> > > >
> > > > booting unprivileged ubuntu trusty and vivid container I get the
> > > > following messaged when shutting them down:
> > > >
> > > > umount: /dev/zero: block devices are not permitted on filesystem
> > >
> > > Apparently this is an error emitted by umount.c in util-linux when it
> > > gets a -EACCES. My guess is that your underlying fs has real blockdevs,
> > > and the fs is MS_NODEV, and umount is somehow finding that unmounting
> > > the file mounted over those files would violate MS_NODEV. I say somehow
> > > bc I don't see the code doing that check.
> > >
> > > Though it's also possible that umount is misdiagnosing the EACCES.
> > > Just to be sure, you could try booting the container without apparmor:
> > >
> > > lxc.aa_profile = unconfined
> > >
> > > and see if that still does it.
> > That won't help as I'm on an Archlinux box which does not use Apparmor and I
> > did not compile it into my kernel. My lxc version is 1.1, cgmanger 0.35, lxcfs
> > 0.5
>
> Ah, well that's just as helpful :)
>
Haha, nice :)
> Which kernel version are you using?
>
3.18.5 downloaded and compiled directly from kernel.org with userns
support compiled in. Additionally, I compiled in (not as module!) btrfs,
FUSE and Overlay.
> > At least the output of findmnt does not show any "nodev" for /dev/sda2:
>
> Is this from the host or container? (looks like the host; if it is,
> can you show /proc/1/mountinfo in the container?)
>
This was from the host side. I will use Debian wheezy as an example as
it is currently up, it's the same with Ubuntu Trusty and Oracle.
Containers with systemd as init running unprivileged are not a problem
because they use lxcfs's fuse fs:
Here is the output of /proc/1/mountinfo from an unprivileged Debian
Wheezy container:
root at wheezy:~# cat /proc/1/mountinfo
138 139 0:17 /@/home/chb/.local/share/lxc/wheezy/rootfs / rw,relatime master:1 - btrfs /dev/sda2 rw,compress-force=lzo,space_cache,autodefrag
247 138 0:82 / /dev rw,nodev,relatime - tmpfs none rw,size=100k,mode=755,uid=100000,gid=100000
248 138 0:84 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
249 248 0:84 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
250 248 0:84 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
251 138 0:85 / /sys rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
252 251 0:85 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs rw
253 252 0:85 / /sys/devices/virtual/net rw,nodev,relatime - sysfs sysfs rw
254 253 0:85 /devices/virtual/net /sys/devices/virtual/net rw,nosuid,nodev,noexec,relatime - sysfs sysfs rw
255 252 0:40 / /sys/fs/fuse/connections rw,relatime master:28 - fusectl fusectl rw
256 247 0:5 /console /dev/console rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
257 247 0:5 /full /dev/full rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
258 247 0:5 /null /dev/null rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
259 247 0:5 /random /dev/random rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
260 247 0:5 /tty /dev/tty rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
261 247 0:5 /urandom /dev/urandom rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
262 247 0:5 /zero /dev/zero rw,nosuid,relatime master:2 - devtmpfs dev rw,size=4035240k,nr_inodes=1008810,mode=755
263 252 0:86 / /sys/fs/cgroup rw,nodev,relatime - tmpfs cgroup rw,size=12k,mode=755,uid=100000,gid=100000
264 263 0:23 /cgmanager /sys/fs/cgroup/cgmanager rw master:8 - tmpfs tmpfs rw,mode=755
265 263 0:87 / /sys/fs/cgroup rw,nodev,relatime - tmpfs none rw,size=4k,mode=755,uid=100000,gid=100000
266 265 0:23 /cgmanager /sys/fs/cgroup/cgmanager rw master:8 - tmpfs tmpfs rw,mode=755
267 256 0:11 /6 /dev/console rw,nosuid,noexec,relatime master:4 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
140 247 0:88 / /dev/pts rw,relatime - devpts devpts rw,gid=100005,mode=620,ptmxmode=666
141 247 0:88 /0 /dev/tty1 rw,relatime - devpts devpts rw,gid=100005,mode=620,ptmxmode=666
142 247 0:88 /1 /dev/tty2 rw,relatime - devpts devpts rw,gid=100005,mode=620,ptmxmode=666
143 247 0:88 /2 /dev/tty3 rw,relatime - devpts devpts rw,gid=100005,mode=620,ptmxmode=666
144 247 0:88 /3 /dev/tty4 rw,relatime - devpts devpts rw,gid=100005,mode=620,ptmxmode=666
145 138 0:89 / /run rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=807664k,mode=755,uid=100000,gid=100000
146 145 0:90 / /run/lock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=5120k,uid=100000,gid=100000
147 145 0:91 / /run/shm rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=1615320k,uid=100000,gid=100000
> I was wondering whether
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3e1866410f11356a9fd869beb3e95983dc79c067
> or
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=9566d6742852c527bf5af38af5cbb878dad75705
> could be involved, though looking more closely i guess probably not.
>
> > [chb at conventiont ~]$ findmnt
> > TARGET SOURCE FSTYPE OPTIONS
> > / /dev/sda2[/@] btrfs rw,relatime,compress-force=lzo,space_cache,autodefrag
> > ├─/proc proc proc rw,nosuid,nodev,noexec,relatime
> > │ └─/proc/sys/fs/binfmt_misc systemd-1 autofs rw,relatime,fd=28,pgrp=1,timeout=300,minproto=5,maxproto=5,direct
> > │ └─/proc/sys/fs/binfmt_misc binfmt_misc binfmt_misc rw,relatime
> > ├─/sys sys sysfs rw,nosuid,nodev,noexec,relatime
> > │ ├─/sys/kernel/security securityfs securityfs rw,nosuid,nodev,noexec,relatime
> > │ ├─/sys/fs/cgroup tmpfs tmpfs rw,mode=755
> > │ │ ├─/sys/fs/cgroup/systemd cgroup cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
> > │ │ ├─/sys/fs/cgroup/cpu,cpuacct cgroup cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct
> > │ │ ├─/sys/fs/cgroup/devices cgroup cgroup rw,nosuid,nodev,noexec,relatime,devices
> > │ │ ├─/sys/fs/cgroup/freezer cgroup cgroup rw,nosuid,nodev,noexec,relatime,freezer
> > │ │ ├─/sys/fs/cgroup/hugetlb cgroup cgroup rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
> > │ │ ├─/sys/fs/cgroup/cpuset cgroup cgroup rw,nosuid,nodev,noexec,relatime,cpuset,clone_children
> > │ │ ├─/sys/fs/cgroup/blkio cgroup cgroup rw,nosuid,nodev,noexec,relatime,blkio
> > │ │ ├─/sys/fs/cgroup/memory cgroup cgroup rw,nosuid,nodev,noexec,relatime,memory
> > │ │ ├─/sys/fs/cgroup/debug cgroup cgroup rw,nosuid,nodev,noexec,relatime,debug
> > │ │ ├─/sys/fs/cgroup/net_cls cgroup cgroup rw,nosuid,nodev,noexec,relatime,net_cls
> > │ │ └─/sys/fs/cgroup/perf_event cgroup cgroup rw,nosuid,nodev,noexec,relatime,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
> > │ ├─/sys/fs/pstore pstore pstore rw,nosuid,nodev,noexec,relatime
> > │ ├─/sys/firmware/efi/efivars efivarfs efivarfs rw,nosuid,nodev,noexec,relatime
> > │ ├─/sys/kernel/debug debugfs debugfs rw,relatime
> > │ ├─/sys/fs/fuse/connections fusectl fusectl rw,relatime
> > │ └─/sys/kernel/config configfs configfs rw,relatime
> > ├─/dev dev devtmpfs rw,nosuid,relatime,size=4035240k,nr_inodes=1008810,mode=755
> > │ ├─/dev/shm tmpfs tmpfs rw,nosuid,nodev
> > │ ├─/dev/pts devpts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
> > │ ├─/dev/hugepages hugetlbfs hugetlbfs rw,relatime
> > │ └─/dev/mqueue mqueue mqueue rw,relatime
> > ├─/run run tmpfs rw,nosuid,nodev,relatime,mode=755
> > │ └─/run/user/1000 tmpfs tmpfs rw,nosuid,nodev,relatime,size=807664k,mode=700,uid=1000,gid=1000
> > ├─/tmp tmpfs tmpfs rw
> > ├─/boot /dev/sda1 vfat rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro
> > └─/var/lib/docker/btrfs /dev/sda2[/@/var/lib/docker/btrfs]
> >
> > >
> > >
> > > > umount: /dev/urandom: block devices are not permitted on filesystem
> > > > umount: /dev/tty: block devices are not permitted on filesystem
> > > > umount: /dev/random: block devices are not permitted on filesystem
> > > > umount: /dev/null: block devices are not permitted on filesystem
> > > > umount: /dev/full: block devices are not permitted on filesystem
> > > > umount: /dev/console: block devices are not permitted on filesystem
> > > >
> > > > that goes for basically all device bind-mounts:
> > > >
> > > > umount: /dev/fb0: block devices are not permitted on filesystem
> > > > umount: /dev/video0: block devices are not permitted on filesystem
> > > > umount: /dev/dri: block devices are not permitted on filesystem
> > > > umount: /dev/snd: block devices are not permitted on filesystem
> > > > umount: /dev/zero: block devices are not permitted on filesystem
> > > > umount: /dev/urandom: block devices are not permitted on filesystem
> > > > umount: /dev/tty: block devices are not permitted on filesystem
> > > > umount: /dev/random: block devices are not permitted on filesystem
> > > > umount: /dev/null: block devices are not permitted on filesystem
> > > > umount: /dev/full: block devices are not permitted on filesystem
> > > > umount: /dev/console: block devices are not permitted on filesystem
> > > >
> > > > Can someone explain this?
> > > >
> > > > Best,
> > > > Christian
> > > > _______________________________________________
> > > > lxc-users mailing list
> > > > lxc-users at lists.linuxcontainers.org
> > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> > >
> >
> >
> > > _______________________________________________
> > > lxc-users mailing list
> > > lxc-users at lists.linuxcontainers.org
> > > http://lists.linuxcontainers.org/listinfo/lxc-users
> >
Christian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20150201/0f2d0c2f/attachment.sig>
More information about the lxc-users
mailing list