[lxc-users] Cannot unlink/rename over host-level file that has been bind-mounted out of a container
S.Çağlar Onur
caglar at 10ur.org
Mon Dec 15 21:31:03 UTC 2014
Sounds like overlayfs is what you are looking for?
# mount -t overlayfs overlayfs ~/rootfs/etc/ -o
rw,upperdir=~/custom,lowerdir=/etc
# rm ~/rootfs/etc/passwd
# ls /etc/passwd
/etc/passwd
# ls custom/ -al
total 8
drwxr-xr-x 2 root root 4096 Dec 15 16:29 .
drwx------ 12 root root 4096 Dec 15 16:29 ..
lrwxrwxrwx 1 root root 18 Dec 15 16:29 passwd -> (overlay-whiteout)
On Mon, Dec 15, 2014 at 3:30 PM, Barry Jaspan <barry.jaspan at acquia.com> wrote:
> On Mon, Dec 15, 2014 at 2:52 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
> wrote:
>>
>> /other_file is bind-mounted *over* /empty. So /empty is busy.
>
>
> Okay, I get it. The inode which is the host's /empty is being used as a
> mount-point. It makes sense that being a mount-point makes the inode busy.
>
> My goal is to use the host's /etc inside the container, but to replace
> /etc/passwd+group+shadow within the container with unique per-container
> versions, while still allowing commands like useradd/chsh/etc, which unlink
> /etc/passwd, to work. My current work-around is to create a hardlink farm
> of /etc to another path and replace passwd+group+shadow with empty files:
>
> cp -al /etc /etc_farm
> rm /etc_farm/{passwd,group,shadow}
> touch /etc_farm/{passwd,group,shadow}
>
> I can then mount /etc_farm as the container's /etc, and mount a unique
> per-container passwd+group+shadow over the container's /etc versions, and
> the host's /etc/passwd+group+shadow are not EBUSY. This works fine. The
> downside is that any changes to the host's /etc are not reflected in
> /etc_farm unless I regenerate /etc_farm.
>
> Does anyone have another suggested approach?
>
> Barry
>
>
>>
>>
>> > host# ls -li /empty /other_file
>> > 57876 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty
>> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file
>> > host# lxc-execute -n test -f test.config -- ls -li /empty /other_file
>> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty
>> > 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file
>> >
>> > I would expect the host's /other_file to be busy, but not the host's
>> > /empty.
>> >
>> > Barry
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > > It is over-mounted. So it
>> > > is busy.
>> > >
>> > > Quoting Barry Jaspan (barry.jaspan at acquia.com):
>> > > > I have reduced the problem I described in my previous message to a
>> > > > much
>> > > > smaller test case. Here is an LXC config file that bind-mounts a
>> > > > single
>> > > > file, /other_file, from the host's root filesystem on top of another
>> > > file,
>> > > > /empty, also from the host's root filesystem:
>> > > >
>> > > > lxc.console = none
>> > > > lxc.rootfs = /
>> > > > lxc.mount.entry=/other_file empty none rw,bind 0 0
>> > > >
>> > > > While the container is running, the host's /empty cannot be
>> > > > unlinked:
>> > > >
>> > > > host# rm /empty /other_file
>> > > > host# touch /empty /other_file
>> > > > host# lxc-execute -n test -f test.config -- sleep 30 &
>> > > > [1] 2419
>> > > > host# strace -o rm.out rm /empty
>> > > > rm: cannot remove `/empty': Device or resource busy
>> > > > host# grep EBUSY rm.out
>> > > > unlinkat(AT_FDCWD, "/empty", 0) = -1 EBUSY (Device or
>> > > > resource
>> > > busy)
>> > > > host# fg
>> > > > lxc-execute -n test -f test.config -- sleep 30
>> > > > ^C
>> > > > host# rm /empty
>> > > > host#
>> > > >
>> > > > Can anyone explain why the host's /empty cannot be unlinked even
>> > > > though
>> > > it
>> > > > is bind-mounted out of the running container? Why is it "busy"?
>> > > >
>> > > > Thanks,
>> > > > Barry
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Dec 8, 2014 at 5:53 PM, Barry Jaspan
>> > > > <barry.jaspan at acquia.com>
>> > > > wrote:
>> > > > >
>> > > > > I'm using LXC on Ubuntu 12.04 (Precise) on EC2. I am creating a
>> > > container
>> > > > > that uses the host's root filesystem. In the lxc config, I am
>> > > bind-mounting
>> > > > > a different passwd file on top of /etc/passwd:
>> > > > >
>> > > > > lxc.mount.entry=/container/passwd etc/passwd none rw,bind 0 0
>> > > > >
>> > > > > As expected, /etc/passwd on the host and /etc/passwd in the
>> > > > > container
>> > > are
>> > > > > different inodes:
>> > > > >
>> > > > > host# ls -i /etc/passwd
>> > > > > 58046 /etc/passwd
>> > > > > host# ls -i /container/passwd
>> > > > > 287145 /container/passwd
>> > > > >
>> > > > > container# ls -i /etc/passwd
>> > > > > 287145 /etc/passwd
>> > > > >
>> > > > > What I did not expect is that now it is not possible to unlink
>> > > /etc/passwd
>> > > > > at the host level (or inside the container, although I don't care
>> > > > > about
>> > > > > that):
>> > > > >
>> > > > > host# rm /etc/passwd
>> > > > > rm: cannot remove `/etc/passwd': Device or resource busy
>> > > > > host# chsh -s /bin/false root
>> > > > > chsh: failure while writing changes to /etc/passwd
>> > > > >
>> > > > > strace confirms that unlinkat() and rename() are returning EBUSY.
>> > > > > (chsh
>> > > > > creates a replacement for /etc/passwd and then rename()'s it into
>> > > place.)
>> > > > >
>> > > > > So, somehow, the fact that the host's /etc/passwd is *not* present
>> > > > > in
>> > > the
>> > > > > container marks its inode as busy. I do not think it is as simple
>> > > > > as
>> > > "an
>> > > > > inode that has been bind-mounted out of a namespace is busy"
>> > > > > because a
>> > > > > simple bind-mount test case does not reproduce it:
>> > > > >
>> > > > > # echo foo > foo
>> > > > > # touch bar
>> > > > > # mount --bind ./foo ./bar
>> > > > > # cat bar
>> > > > > foo
>> > > > > # touch new
>> > > > > # mv new foo
>> > > > > #
>> > > > >
>> > > > > This seems related to http://lwn.net/Articles/570338/. However, in
>> > > that
>> > > > > article, a file which is bind-mounted into another namespace
>> > > > > causes
>> > > EBUSY
>> > > > > from unlink/rename(). In my case, a file which is *not*
>> > > > > bind-mounted
>> > > into
>> > > > > another namespace is causing EBUSY.
>> > > > >
>> > > > > Can someone explain what is going on?
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Barry
>> > > > >
>> > > > > --
>> > > > > Barry Jaspan
>> > > > > Senior Architect | Acquia <http://acquia.com>
>> > > > > barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>> > > > >
>> > > > > Acquia Dev Cloud: You build killer websites. We do the rest.
>> > > > > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
>> > > > > Acquia ranked #1 Software Vendor on the 2012 Inc 500
>> > > > > <
>> > >
>> > > http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing
>> > > >
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > Barry Jaspan
>> > > > Senior Architect | Acquia <http://acquia.com>
>> > > > barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>> > > >
>> > > > Acquia Dev Cloud: You build killer websites. We do the rest.
>> > > > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
>> > > > Acquia ranked #1 Software Vendor on the 2012 Inc 500
>> > > > <
>> > >
>> > > http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing
>> > > >
>> > >
>> > > > _______________________________________________
>> > > > lxc-users mailing list
>> > > > lxc-users at lists.linuxcontainers.org
>> > > > http://lists.linuxcontainers.org/listinfo/lxc-users
>> > >
>> > > _______________________________________________
>> > > lxc-users mailing list
>> > > lxc-users at lists.linuxcontainers.org
>> > > http://lists.linuxcontainers.org/listinfo/lxc-users
>> >
>> >
>> >
>> > --
>> > Barry Jaspan
>> > Senior Architect | Acquia <http://acquia.com>
>> > barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>> >
>> > Acquia Dev Cloud: You build killer websites. We do the rest.
>> > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
>> > Acquia ranked #1 Software Vendor on the 2012 Inc 500
>> >
>> > <http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing>
>>
>> > _______________________________________________
>> > lxc-users mailing list
>> > lxc-users at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-users
>>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
>
> --
> Barry Jaspan
> Senior Architect | Acquia
> barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>
> Acquia Dev Cloud: You build killer websites. We do the rest.
> Acquia ranked #1 Software Vendor on the 2012 Inc 500
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
--
S.Çağlar Onur <caglar at 10ur.org>
More information about the lxc-users
mailing list