<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Dec 15, 2014 at 2:52 PM, Serge Hallyn <span dir="ltr"><<a href="mailto:serge.hallyn@ubuntu.com" target="_blank">serge.hallyn@ubuntu.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">/other_file is bind-mounted *over* /empty. So /empty is busy.<br></blockquote><div><br></div><div>Okay, I get it. The inode which is the host's /empty is being used as a mount-point. It makes sense that being a mount-point makes the inode busy.</div><div><br></div><div>My goal is to use the host's /etc inside the container, but to replace /etc/passwd+group+shadow within the container with unique per-container versions, while still allowing commands like useradd/chsh/etc, which unlink /etc/passwd, to work. My current work-around is to create a hardlink farm of /etc to another path and replace passwd+group+shadow with empty files:</div><div><br></div><div>cp -al /etc /etc_farm</div><div>rm /etc_farm/{passwd,group,shadow}</div><div>touch /etc_farm/{passwd,group,shadow}</div><div><br></div><div>I can then mount /etc_farm as the container's /etc, and mount a unique per-container passwd+group+shadow over the container's /etc versions, and the host's /etc/passwd+group+shadow are not EBUSY. This works fine. The downside is that any changes to the host's /etc are not reflected in /etc_farm unless I regenerate /etc_farm.</div><div><br></div><div>Does anyone have another suggested approach?</div><div><br></div><div>Barry</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class=""><div class="h5"><br>
> host# ls -li /empty /other_file<br>
> 57876 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty<br>
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file<br>
> host# lxc-execute -n test -f test.config -- ls -li /empty /other_file<br>
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /empty<br>
> 58108 -rw-r--r-- 1 root root 0 Dec 15 19:26 /other_file<br>
><br>
> I would expect the host's /other_file to be busy, but not the host's /empty.<br>
><br>
> Barry<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> > It is over-mounted. So it<br>
> > is busy.<br>
> ><br>
> > Quoting Barry Jaspan (<a href="mailto:barry.jaspan@acquia.com">barry.jaspan@acquia.com</a>):<br>
> > > I have reduced the problem I described in my previous message to a much<br>
> > > smaller test case. Here is an LXC config file that bind-mounts a single<br>
> > > file, /other_file, from the host's root filesystem on top of another<br>
> > file,<br>
> > > /empty, also from the host's root filesystem:<br>
> > ><br>
> > > lxc.console = none<br>
> > > lxc.rootfs = /<br>
> > > lxc.mount.entry=/other_file empty none rw,bind 0 0<br>
> > ><br>
> > > While the container is running, the host's /empty cannot be unlinked:<br>
> > ><br>
> > > host# rm /empty /other_file<br>
> > > host# touch /empty /other_file<br>
> > > host# lxc-execute -n test -f test.config -- sleep 30 &<br>
> > > [1] 2419<br>
> > > host# strace -o rm.out rm /empty<br>
> > > rm: cannot remove `/empty': Device or resource busy<br>
> > > host# grep EBUSY rm.out<br>
> > > unlinkat(AT_FDCWD, "/empty", 0) = -1 EBUSY (Device or resource<br>
> > busy)<br>
> > > host# fg<br>
> > > lxc-execute -n test -f test.config -- sleep 30<br>
> > > ^C<br>
> > > host# rm /empty<br>
> > > host#<br>
> > ><br>
> > > Can anyone explain why the host's /empty cannot be unlinked even though<br>
> > it<br>
> > > is bind-mounted out of the running container? Why is it "busy"?<br>
> > ><br>
> > > Thanks,<br>
> > > Barry<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > On Mon, Dec 8, 2014 at 5:53 PM, Barry Jaspan <<a href="mailto:barry.jaspan@acquia.com">barry.jaspan@acquia.com</a>><br>
> > > wrote:<br>
> > > ><br>
> > > > I'm using LXC on Ubuntu 12.04 (Precise) on EC2. I am creating a<br>
> > container<br>
> > > > that uses the host's root filesystem. In the lxc config, I am<br>
> > bind-mounting<br>
> > > > a different passwd file on top of /etc/passwd:<br>
> > > ><br>
> > > > lxc.mount.entry=/container/passwd etc/passwd none rw,bind 0 0<br>
> > > ><br>
> > > > As expected, /etc/passwd on the host and /etc/passwd in the container<br>
> > are<br>
> > > > different inodes:<br>
> > > ><br>
> > > > host# ls -i /etc/passwd<br>
> > > > 58046 /etc/passwd<br>
> > > > host# ls -i /container/passwd<br>
> > > > 287145 /container/passwd<br>
> > > ><br>
> > > > container# ls -i /etc/passwd<br>
> > > > 287145 /etc/passwd<br>
> > > ><br>
> > > > What I did not expect is that now it is not possible to unlink<br>
> > /etc/passwd<br>
> > > > at the host level (or inside the container, although I don't care about<br>
> > > > that):<br>
> > > ><br>
> > > > host# rm /etc/passwd<br>
> > > > rm: cannot remove `/etc/passwd': Device or resource busy<br>
> > > > host# chsh -s /bin/false root<br>
> > > > chsh: failure while writing changes to /etc/passwd<br>
> > > ><br>
> > > > strace confirms that unlinkat() and rename() are returning EBUSY. (chsh<br>
> > > > creates a replacement for /etc/passwd and then rename()'s it into<br>
> > place.)<br>
> > > ><br>
> > > > So, somehow, the fact that the host's /etc/passwd is *not* present in<br>
> > the<br>
> > > > container marks its inode as busy. I do not think it is as simple as<br>
> > "an<br>
> > > > inode that has been bind-mounted out of a namespace is busy" because a<br>
> > > > simple bind-mount test case does not reproduce it:<br>
> > > ><br>
> > > > # echo foo > foo<br>
> > > > # touch bar<br>
> > > > # mount --bind ./foo ./bar<br>
> > > > # cat bar<br>
> > > > foo<br>
> > > > # touch new<br>
> > > > # mv new foo<br>
> > > > #<br>
> > > ><br>
> > > > This seems related to <a href="http://lwn.net/Articles/570338/" target="_blank">http://lwn.net/Articles/570338/</a>. However, in<br>
> > that<br>
> > > > article, a file which is bind-mounted into another namespace causes<br>
> > EBUSY<br>
> > > > from unlink/rename(). In my case, a file which is *not* bind-mounted<br>
> > into<br>
> > > > another namespace is causing EBUSY.<br>
> > > ><br>
> > > > Can someone explain what is going on?<br>
> > > ><br>
> > > > Thanks,<br>
> > > ><br>
> > > > Barry<br>
> > > ><br>
> > > > --<br>
> > > > Barry Jaspan<br>
> > > > Senior Architect | Acquia <<a href="http://acquia.com" target="_blank">http://acquia.com</a>><br>
> > > > <a href="mailto:barry.jaspan@acquia.com">barry.jaspan@acquia.com</a> | (c) <a href="tel:617.905.2208" value="+16179052208">617.905.2208</a> | (w) <a href="tel:781-313-8298" value="+17813138298">781-313-8298</a><br>
> > > ><br>
> > > > Acquia Dev Cloud: You build killer websites. We do the rest.<br>
> > > > <<a href="http://www.acquia.com/dev-cloud" target="_blank">http://www.acquia.com/dev-cloud</a>> <<a href="http://acquia.com/dev-cloud" target="_blank">http://acquia.com/dev-cloud</a>><br>
> > > > Acquia ranked #1 Software Vendor on the 2012 Inc 500<br>
> > > > <<br>
> > <a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank">http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing</a><br>
> > ><br>
> > > ><br>
> > > ><br>
> > ><br>
> > > --<br>
> > > Barry Jaspan<br>
> > > Senior Architect | Acquia <<a href="http://acquia.com" target="_blank">http://acquia.com</a>><br>
> > > <a href="mailto:barry.jaspan@acquia.com">barry.jaspan@acquia.com</a> | (c) <a href="tel:617.905.2208" value="+16179052208">617.905.2208</a> | (w) <a href="tel:781-313-8298" value="+17813138298">781-313-8298</a><br>
> > ><br>
> > > Acquia Dev Cloud: You build killer websites. We do the rest.<br>
> > > <<a href="http://www.acquia.com/dev-cloud" target="_blank">http://www.acquia.com/dev-cloud</a>> <<a href="http://acquia.com/dev-cloud" target="_blank">http://acquia.com/dev-cloud</a>><br>
> > > Acquia ranked #1 Software Vendor on the 2012 Inc 500<br>
> > > <<br>
> > <a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank">http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing</a><br>
> > ><br>
> ><br>
> > > _______________________________________________<br>
> > > lxc-users mailing list<br>
> > > <a href="mailto:lxc-users@lists.linuxcontainers.org">lxc-users@lists.linuxcontainers.org</a><br>
> > > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
> ><br>
> > _______________________________________________<br>
> > lxc-users mailing list<br>
> > <a href="mailto:lxc-users@lists.linuxcontainers.org">lxc-users@lists.linuxcontainers.org</a><br>
> > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
><br>
><br>
><br>
> --<br>
> Barry Jaspan<br>
> Senior Architect | Acquia <<a href="http://acquia.com" target="_blank">http://acquia.com</a>><br>
> <a href="mailto:barry.jaspan@acquia.com">barry.jaspan@acquia.com</a> | (c) <a href="tel:617.905.2208" value="+16179052208">617.905.2208</a> | (w) <a href="tel:781-313-8298" value="+17813138298">781-313-8298</a><br>
><br>
> Acquia Dev Cloud: You build killer websites. We do the rest.<br>
> <<a href="http://www.acquia.com/dev-cloud" target="_blank">http://www.acquia.com/dev-cloud</a>> <<a href="http://acquia.com/dev-cloud" target="_blank">http://acquia.com/dev-cloud</a>><br>
> Acquia ranked #1 Software Vendor on the 2012 Inc 500<br>
> <<a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank">http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing</a>><br>
<br>
> _______________________________________________<br>
> lxc-users mailing list<br>
> <a href="mailto:lxc-users@lists.linuxcontainers.org">lxc-users@lists.linuxcontainers.org</a><br>
> <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
<br>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a></div></div></blockquote></div><br clear="all"><div><br></div>-- <br><div class="gmail_signature">Barry Jaspan<br><font size="1"><span style="color:rgb(102,102,102)">Senior Architect | </span><a style="color:rgb(102,102,102)" href="http://acquia.com" target="_blank">Acquia</a><br style="color:rgb(102,102,102)"><a style="color:rgb(102,102,102)" href="mailto:barry.jaspan@acquia.com" target="_blank">barry.jaspan@acquia.com</a><span style="color:rgb(102,102,102)"> | (c) 617.905.2208 | (w) 781-313-8298</span><br style="color:rgb(102,102,102)"><br style="color:rgb(102,102,102)"><font><a href="http://www.acquia.com/dev-cloud" target="_blank">Acquia Dev Cloud: You build killer websites. We do the rest.</a><a href="http://acquia.com/dev-cloud" target="_blank"></a></font></font><br><a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank"><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);font-weight:bold;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;background-color:transparent">Acquia</span><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;background-color:transparent"> ranked #1 Software Vendor on the 2012 Inc 500</span></a><br><br></div>
</div></div>