[Lxc-users] read only rootfs

[Michael H. Warfield]

On Tue, 2011-07-19 at 17:28 -0500, C Anthony Risinger wrote: 
> On Tue, Jul 19, 2011 at 4:17 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> > On Tue, 2011-07-19 at 15:32 -0500, Serge E. Hallyn wrote:
> >> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> >> > On Tue, 2011-07-19 at 13:34 -0500, Serge E. Hallyn wrote:
> >> > > Quoting C Anthony Risinger (anthony at xtfx.me):
> >> > > > there it would seem.  however, while i could *maybe* see the rootfs
> >> > > > being an unconditional slave, i would NOT want to see any lxc
> >> > > > default/enforcement preventing container -> host propagation on a
> >> > > > globally recursive scale.  im of the opinion that the implementor
> >> > > > should decide the best tactic ... especially in light of the fact the
> >> > > > one distro may not even have the same problems as say
> >> > > > ubutnu/fedora/etc because they keep mount points private by default.
> >> >
> >> > > Good point.  (I don't see it on ubuntu either fwiw)  Perhaps there
> >> > > should be a toggle in the per-container config file?
> >> >
> >> > Quick question.
> >> >
> >> > Is there any way to test for these flags (SHARED, PRIVATE, SLAVE)?  I
> >> > don't see them showing up anywhere from mount, in proc mounts or
> >> > mountstats.  How do you check to see if they are set?
> >
> >> /proc/self/mountinfo is supposed to tell that.  i.e. if you do
> >> a --make-shared on /mnt, it'll show 'shared' next to the /mnt entry.
> >> (I say 'is supposed to' bc --make-rslave just shows nothing, but
> >> maybe that's bc the way i did it it wasn't a slave to anything,
> >> so it was actually private)
> >
> > Ok...  This just gets weirder.
> >
> > For giggles, I set my /srv partition (where all my VM's are located) to
> > "shared".  Now. the first machine starts up fine but the second one,
> > Plover, and all subsequent ones blow up with this:
> >
> > [root at forest ~]# lxc-start --name Plover
> > lxc-start: Invalid argument - pivot_root syscall failed
> > lxc-start: failed to setup pivot root
> > lxc-start: failed to set rootfs for 'Plover'
> > lxc-start: failed to setup the container
> > lxc-start: invalid sequence number 1. expected 2
> > lxc-start: failed to spawn 'Plover'
> > lxc-start: Device or resource busy - failed to remove cgroup '/sys/fs/cgroup/systemd/Plover'
> >
> > And mount -t devpts shows ALL the devpts mounts for all the attempted
> > VM's.  Ok...  Guess that wasn't a good idea.
> >
> > But...  I got this for the root system on Alcove.
> >
> > 106 55 8:17 /lxc/private/Alcove / rw,relatime master:1 - ext4 /dev/sdb1 rw,barrier=1,data=ordered
> >
> > Ok...  That now says "master:1".  Not sure what it signifies...
> >
> > Shut him down and changed /srv to be slave and all the containers come
> > up but the remount still propagates back.  Changed ran --make-rslave on
> > it and no influence.  Seems like we're missing a piece of the puzzle
> > here.
> 
> maybe not the best context for this response, but i wanted to point
> out one thing that confused me for awhile since it might be related
> ...
> 
> ... that fact that the shared/slave context only exists with BOTH
> sides are mount points.  eg. if DIR is only a directory:

> mount --bind ./DIR ./TARGET

> ... it will never propagate mounts to TARGET (AFAICT), and does not
> respond to --make-* ... before OR after the --bind.  in order to get
> propagation, one must:

> mount --bind ./DIR ./DIR
> mount --make-shared ./DIR
> mount --bind ./DIR ./TARGET
> [mount --make-slave ./TARGET]

Wow.  Ouch.

That is very interesting.  Painfully interesting.

Unfortunately, it still didn't work.

On the host:

[root at forest lxc]# mount --bind private/Alcove private/Alcove
[root at forest lxc]# mount --make-share private/Alcove
[root at forest lxc]# mount --bind private/Alcove root/Alcove
[root at forest lxc]# mount --make-slave root/Alcove 
[root at forest mhw]# grep Alcove /proc/self/mountinfo 
58 45 8:17 /lxc/private/Alcove /srv/lxc/private/Alcove rw,relatime shared:1 - ext4 /dev/sdb1 rw,barrier=1,data=ordered
59 45 8:17 /lxc/private/Alcove /srv/lxc/root/Alcove rw,relatime master:1 - ext4 /dev/sdb1 rw,barrier=1,data=ordered

Ok...  I see the shared and the master:1 appears to be the slave.

[root at forest mhw]# mount -t devpts
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)

/dev/pts is rw and normal at this point.

In the Alcove config file it has:

lxc.rootfs = /srv/lxc/root/Alcove

Run: lxc-start --name Alcove

Fires the container up.

Now...  In the container...

[root at alcove mhw]# cat /proc/self/mountinfo 
110 61 8:17 /lxc/private/Alcove / rw,relatime master:1 - ext4 /dev/sdb1 rw,barrier=1,data=ordered
111 110 0:10 /7 /dev/console rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
112 110 0:10 /1 /dev/tty1 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
113 110 0:10 /2 /dev/tty2 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
114 110 0:10 /3 /dev/tty3 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
115 110 0:10 /4 /dev/tty4 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
116 110 0:10 /5 /dev/tty5 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
117 110 0:10 /6 /dev/tty6 rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
63 110 0:45 / /dev/pts rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
64 110 0:44 / /proc rw,nosuid,nodev,noexec,relatime - proc none rw
65 110 0:46 / /sys rw,nosuid,nodev,noexec,relatime - sysfs none rw
66 64 0:14 / /proc/bus/usb rw,relatime - usbfs /proc/bus/usb rw
67 63 0:10 / /dev/pts rw,relatime - devpts devpts rw,mode=600,ptmxmode=666
68 64 0:36 / /proc/sys/fs/binfmt_misc rw,relatime - binfmt_misc none rw
[root at alcove mhw]# mount -t devpts
devpts on /dev/console type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty1 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty2 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty3 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty4 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty5 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/tty6 type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)

On the host...

[root at forest mhw]# mount -t devpts
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)

Still good.  Now, on the container...

[root at alcove mhw]# mount -o remount,ro /dev/pts

Meanwhile, back at the ranch...

[root at forest mhw]# mount -t devpts
devpts on /dev/pts type devpts (ro,relatime,mode=600,ptmxmode=666)

Ah, bletch.

[root at alcove mhw]# mount -o remount,rw /dev/pts

[root at forest mhw]# mount -t devpts
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=666)

No joy.  Seemed like the right idea, rather convoluted but heading in
the right direction.  Close but no cigar.

> ... this tripped me up for awhile as it seemed like the semantics were changing.

Thanks!

> C Anthony

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20110719/7611c30c/attachment.pgp>