[lxc-users] lxc-execute with read-only rootfs

Thu Dec 19 10:29:31 UTC 2013

On Wed, Dec 18, 2013 at 06:29:57PM -0800, Antonin Bas wrote:
> Thanks Cal. I will look into this.
> However, for now, I have found a quick fix. I just added "lxc.pivotdir
> = /mnt" to my config file. The pivotdir is used when pivoting the
> original root file system. It seems that in older versions of lxc, the
> default value was /mnt but this was changed since (in my case it was
> /usr/lib/x86_64-linux-gnu/
> lxc/lxc_putold by default). Nothing is actually written in this
> directory, so it can work even in read-only mode. However, if this
> location does not exist lxc will try to create it, which will fail.
> mnt/ exists so no problem.
> I hope I did not break anything by changing it to /mnt. So far it
> seems to be working.

You should be fine.

The reason we set it to something other than our previous default of
/mnt was issues when you had some mounts defined in the container's
fstab that were also using /mnt.
Apparently in your setup that's not the case so you won't have any problem.

> Note that I still have the following error:
> 
> lxc-execute: Read-only file system - error unlinking
> /usr/lib/x86_64-linux-gnu/lxc/
> dev/kmsg
> 
> but it does not provide the container from spawning.
> 
> Antonin
> 
> 2013/12/18 Cal Leeming [Simplicity Media Ltd]
> <cal.leeming at simplicitymedialtd.co.uk>:
> > It looks like something being executed is expecting your rootfs to be
> > writeable, this could be something in lxc-execute or it could be /bin/bash
> > itself (I wouldn't know without strace'ing both or looking at the source).
> >
> > You could try adding the necessary /dev, /proc and /sys mounts, which may be
> > enough to get it to run without error (add the following into your lxc
> > config);
> > http://pastebin.com/ZAcXn926
> >
> > However any applications expecting a write-able FS are going to fail hard -
> > for example, anything relying on /tmp, or /var/log, and numerous other
> > places. You could in theory bind mount a ramfs to those locations, but it
> > would be easier, in my opinion, to use a stacked file system.
> >
> > Here is an example of using overlayfs;
> > http://askubuntu.com/questions/109413/how-do-i-use-overlayfs
> >
> > In short, you'd build your container rootfs and make whatever changes you
> > wanted to make, once you're happy with it you then do;
> >
> > $ mount -t overlayfs -o
> > lowerdir=/your.rootfs.here,upperdir=/your.discarded.changes.here overlayfs
> > /your.new.mount.here
> > lxc.rootfs = /your.new.mount.here
> >
> > There is also a previous discussion on this;
> > http://osdir.com/ml/lxc-chroot-linux-containers/2011-07/msg00019.html
> > https://www.redhat.com/archives/libvirt-users/2011-December/msg00024.html
> > http://s3hh.wordpress.com/2011/09/22/sharing-mounts-with-a-container/
> > https://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg02190.html
> >
> > Hope this helps
> >
> > Cal
> >
> >
> > On Thu, Dec 19, 2013 at 1:51 AM, Antonin Bas <antoninb at stanford.edu> wrote:
> >>
> >> Hi,
> >>
> >> I guess that could do it. But I don't understand while it would not be
> >> possible to have a read-only rootfs (i.e. is what I am seeing the
> >> expected behaviour?).
> >> What would the configuration look like if I decided to use overlayfs?
> >> Is it easy to use with lxc-execute? Because I just want to run an
> >> application, and I don't want to pay the overhead of
> >> lxc-start-ephemeral.
> >>
> >> Thanks,
> >>
> >> Antonin
> >>
> >> 2013/12/18 Cal Leeming [Simplicity Media Ltd]
> >> <cal.leeming at simplicitymedialtd.co.uk>:
> >> > Would it not be better to use a stacked file system, such as overlayfs
> >> > or
> >> > aufs, then discard the changes?
> >> >
> >> > Cal
> >> >
> >> >
> >> > On Thu, Dec 19, 2013 at 12:49 AM, Antonin Bas <antoninb at stanford.edu>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I am trying to run an application container with lxc-execute. I am
> >> >> going to run "untrusted" student codes in this container and I want
> >> >> the root file system to be shared with the host but read-only. I
> >> >> thought this would be as easy as using the following configuration
> >> >> file:
> >> >>
> >> >> # Container with new network withtout network devices
> >> >> lxc.utsname = omega
> >> >> lxc.network.type = empty
> >> >> lxc.network.flags = up
> >> >>
> >> >> lxc.rootfs = /tmp/guest/rootfs
> >> >> lxc.mount.entry=/ /tmp/guest/rootfs/ none ro,bind 0 0
> >> >>
> >> >> However, when I run `sudo lxc-execute -n test -f grader.conf
> >> >> --logpriority=DEBUG -- /bin/bash`, I get the following message:
> >> >>
> >> >> lxc-execute: Read-only file system - error unlinking
> >> >> /usr/lib/x86_64-linux-gnu/lxc/dev/kmsg
> >> >>
> >> >> lxc-execute: failed to setup kmsg for 'test'
> >> >> lxc-execute: Read-only file system - failed to create directory
> >> >> '/usr/lib/x86_64-linux-gnu/lxc/lxc_putold'
> >> >>
> >> >> lxc-execute: Read-only file system - failed to create pivotdir
> >> >> '/usr/lib/x86_64-linux-gnu/lxc/lxc_putold'
> >> >> lxc-execute: failed to setup pivot root
> >> >> lxc-execute: failed to set rootfs for 'test'
> >> >> lxc-execute: failed to setup the container
> >> >> lxc-execute: invalid sequence number 1. expected 2
> >> >> lxc-execute: failed to spawn 'test'
> >> >>
> >> >>
> >> >> Is it possible to have some insight on what the problem is here?
> >> >> I am using Ubuntu 13.10, and my lxc is the one from the official repo
> >> >> (1.0.0.alpha1).
> >> >>
> >> >> Thanks you in advance for your help,
> >> >>
> >> >> Antonin
> >> >> _______________________________________________
> >> >> lxc-users mailing list
> >> >> lxc-users at lists.linuxcontainers.org
> >> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > lxc-users mailing list
> >> > lxc-users at lists.linuxcontainers.org
> >> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> _______________________________________________
> >> lxc-users mailing list
> >> lxc-users at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >
> >
> >
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20131219/29f39010/attachment.pgp>