[lxc-users] Cannot unlink/rename over host-level file that has been bind-mounted out of a container

Mon Dec 15 19:34:16 UTC 2014

Interestingly, the host's /other_file is *not* busy; it can be unlinked
outside the container even though it a bind-mounted into the container:

# lxc-execute -n test -f test.config -- sleep 30 &
[1] 16178
# rm /other_file
# fg
lxc-execute -n test -f test.config -- sleep 30
^C
#

This leaves me with no intuition about what it means for the file to be
"busy" due to bind-mounting.

Barry

On Mon, Dec 15, 2014 at 2:29 PM, Barry Jaspan <barry.jaspan at acquia.com>
wrote:
>
> On Mon, Dec 15, 2014 at 2:19 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
> wrote:
>>
>> Since you are using / as the container rootfs, /empty is the same
>> inode in the container and on the host.
>
>
> No, it isn't. Within the container, /empty is the inode of the host's
> /other_file, because /other_file is bind-mounted over /empty:
>
> lxc.mount.entry=/other_file empty none rw,bind 0 0
>
> host# ls -li /empty /other_file
> 57876 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file
> host# lxc-execute -n test -f test.config -- ls -li /empty /other_file
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file
>
> I would expect the host's /other_file to be busy, but not the host's
> /empty.
>
> Barry
>
>
>
>
>
>
>
>> It is over-mounted.  So it
>> is busy.
>>
>> Quoting Barry Jaspan (barry.jaspan at acquia.com):
>> > I have reduced the problem I described in my previous message to a much
>> > smaller test case. Here is an LXC config file that bind-mounts a single
>> > file, /other_file, from the host's root filesystem on top of another
>> file,
>> > /empty, also from the host's root filesystem:
>> >
>> > lxc.console = none
>> > lxc.rootfs = /
>> > lxc.mount.entry=/other_file empty none rw,bind 0 0
>> >
>> > While the container is running, the host's /empty cannot be unlinked:
>> >
>> > host# rm /empty /other_file
>> > host# touch /empty /other_file
>> > host# lxc-execute -n test -f test.config -- sleep 30 &
>> > [1] 2419
>> > host# strace -o rm.out rm /empty
>> > rm: cannot remove `/empty': Device or resource busy
>> > host# grep EBUSY rm.out
>> > unlinkat(AT_FDCWD, "/empty", 0)         = -1 EBUSY (Device or resource
>> busy)
>> > host# fg
>> > lxc-execute -n test -f test.config -- sleep 30
>> > ^C
>> > host# rm /empty
>> > host#
>> >
>> > Can anyone explain why the host's /empty cannot be unlinked even though
>> it
>> > is bind-mounted out of the running container? Why is it "busy"?
>> >
>> > Thanks,
>> > Barry
>> >
>> >
>> >
>> >
>> > On Mon, Dec 8, 2014 at 5:53 PM, Barry Jaspan <barry.jaspan at acquia.com>
>> > wrote:
>> > >
>> > > I'm using LXC on Ubuntu 12.04 (Precise) on EC2. I am creating a
>> container
>> > > that uses the host's root filesystem. In the lxc config, I am
>> bind-mounting
>> > > a different passwd file on top of /etc/passwd:
>> > >
>> > > lxc.mount.entry=/container/passwd etc/passwd none rw,bind 0 0
>> > >
>> > > As expected, /etc/passwd on the host and /etc/passwd in the container
>> are
>> > > different inodes:
>> > >
>> > > host# ls -i /etc/passwd
>> > > 58046 /etc/passwd
>> > > host# ls -i /container/passwd
>> > > 287145 /container/passwd
>> > >
>> > > container# ls -i /etc/passwd
>> > > 287145 /etc/passwd
>> > >
>> > > What I did not expect is that now it is not possible to unlink
>> /etc/passwd
>> > > at the host level (or inside the container, although I don't care
>> about
>> > > that):
>> > >
>> > > host# rm /etc/passwd
>> > > rm: cannot remove `/etc/passwd': Device or resource busy
>> > > host#  chsh -s /bin/false root
>> > > chsh: failure while writing changes to /etc/passwd
>> > >
>> > > strace confirms that unlinkat() and rename() are returning EBUSY.
>> (chsh
>> > > creates a replacement for /etc/passwd and then rename()'s it into
>> place.)
>> > >
>> > > So, somehow, the fact that the host's /etc/passwd is *not* present in
>> the
>> > > container marks its inode as busy. I do not think it is as simple as
>> "an
>> > > inode that has been bind-mounted out of a namespace is busy" because a
>> > > simple bind-mount test case does not reproduce it:
>> > >
>> > > # echo foo > foo
>> > > # touch bar
>> > > # mount --bind ./foo ./bar
>> > > # cat bar
>> > > foo
>> > > # touch new
>> > > # mv new foo
>> > > #
>> > >
>> > > This seems related to http://lwn.net/Articles/570338/. However, in
>> that
>> > > article, a file which is bind-mounted into another namespace causes
>> EBUSY
>> > > from unlink/rename(). In my case, a file which is *not* bind-mounted
>> into
>> > > another namespace is causing EBUSY.
>> > >
>> > > Can someone explain what is going on?
>> > >
>> > > Thanks,
>> > >
>> > > Barry
>> > >
>> > > --
>> > > Barry Jaspan
>> > > Senior Architect | Acquia <http://acquia.com>
>> > > barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>> > >
>> > > Acquia Dev Cloud: You build killer websites. We do the rest.
>> > > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
>> > > Acquia ranked #1 Software Vendor on the 2012 Inc 500
>> > > <
>> http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing
>> >
>> > >
>> > >
>> >
>> > --
>> > Barry Jaspan
>> > Senior Architect | Acquia <http://acquia.com>
>> > barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>> >
>> > Acquia Dev Cloud: You build killer websites. We do the rest.
>> > <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
>> > Acquia ranked #1 Software Vendor on the 2012 Inc 500
>> > <
>> http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing
>> >
>>
>> > _______________________________________________
>> > lxc-users mailing list
>> > lxc-users at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-users
>>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
>
> --
> Barry Jaspan
> Senior Architect | Acquia <http://acquia.com>
> barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298
>
> Acquia Dev Cloud: You build killer websites. We do the rest.
> <http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
> Acquia ranked #1 Software Vendor on the 2012 Inc 500
> <http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing>
>
>

-- 
Barry Jaspan
Senior Architect | Acquia <http://acquia.com>
barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298

Acquia Dev Cloud: You build killer websites. We do the rest.
<http://www.acquia.com/dev-cloud> <http://acquia.com/dev-cloud>
Acquia ranked #1 Software Vendor on the 2012 Inc 500
<http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20141215/fd57d812/attachment.html>