[lxc-users] lxc-execute with read-only rootfs

[Antonin Bas]

Thanks Cal. I will look into this.
However, for now, I have found a quick fix. I just added "lxc.pivotdir
= /mnt" to my config file. The pivotdir is used when pivoting the
original root file system. It seems that in older versions of lxc, the
default value was /mnt but this was changed since (in my case it was
/usr/lib/x86_64-linux-gnu/
lxc/lxc_putold by default). Nothing is actually written in this
directory, so it can work even in read-only mode. However, if this
location does not exist lxc will try to create it, which will fail.
mnt/ exists so no problem.
I hope I did not break anything by changing it to /mnt. So far it
seems to be working.
Note that I still have the following error:

lxc-execute: Read-only file system - error unlinking
/usr/lib/x86_64-linux-gnu/lxc/
dev/kmsg

but it does not provide the container from spawning.

Antonin

2013/12/18 Cal Leeming [Simplicity Media Ltd]
<cal.leeming at simplicitymedialtd.co.uk>:
> It looks like something being executed is expecting your rootfs to be
> writeable, this could be something in lxc-execute or it could be /bin/bash
> itself (I wouldn't know without strace'ing both or looking at the source).
>
> You could try adding the necessary /dev, /proc and /sys mounts, which may be
> enough to get it to run without error (add the following into your lxc
> config);
> http://pastebin.com/ZAcXn926
>
> However any applications expecting a write-able FS are going to fail hard -
> for example, anything relying on /tmp, or /var/log, and numerous other
> places. You could in theory bind mount a ramfs to those locations, but it
> would be easier, in my opinion, to use a stacked file system.
>
> Here is an example of using overlayfs;
> http://askubuntu.com/questions/109413/how-do-i-use-overlayfs
>
> In short, you'd build your container rootfs and make whatever changes you
> wanted to make, once you're happy with it you then do;
>
> $ mount -t overlayfs -o
> lowerdir=/your.rootfs.here,upperdir=/your.discarded.changes.here overlayfs
> /your.new.mount.here
> lxc.rootfs = /your.new.mount.here
>
> There is also a previous discussion on this;
> http://osdir.com/ml/lxc-chroot-linux-containers/2011-07/msg00019.html
> https://www.redhat.com/archives/libvirt-users/2011-December/msg00024.html
> http://s3hh.wordpress.com/2011/09/22/sharing-mounts-with-a-container/
> https://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg02190.html
>
> Hope this helps
>
> Cal
>
>
> On Thu, Dec 19, 2013 at 1:51 AM, Antonin Bas <antoninb at stanford.edu> wrote:
>>
>> Hi,
>>
>> I guess that could do it. But I don't understand while it would not be
>> possible to have a read-only rootfs (i.e. is what I am seeing the
>> expected behaviour?).
>> What would the configuration look like if I decided to use overlayfs?
>> Is it easy to use with lxc-execute? Because I just want to run an
>> application, and I don't want to pay the overhead of
>> lxc-start-ephemeral.
>>
>> Thanks,
>>
>> Antonin
>>
>> 2013/12/18 Cal Leeming [Simplicity Media Ltd]
>> <cal.leeming at simplicitymedialtd.co.uk>:
>> > Would it not be better to use a stacked file system, such as overlayfs
>> > or
>> > aufs, then discard the changes?
>> >
>> > Cal
>> >
>> >
>> > On Thu, Dec 19, 2013 at 12:49 AM, Antonin Bas <antoninb at stanford.edu>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> I am trying to run an application container with lxc-execute. I am
>> >> going to run "untrusted" student codes in this container and I want
>> >> the root file system to be shared with the host but read-only. I
>> >> thought this would be as easy as using the following configuration
>> >> file:
>> >>
>> >> # Container with new network withtout network devices
>> >> lxc.utsname = omega
>> >> lxc.network.type = empty
>> >> lxc.network.flags = up
>> >>
>> >> lxc.rootfs = /tmp/guest/rootfs
>> >> lxc.mount.entry=/ /tmp/guest/rootfs/ none ro,bind 0 0
>> >>
>> >> However, when I run `sudo lxc-execute -n test -f grader.conf
>> >> --logpriority=DEBUG -- /bin/bash`, I get the following message:
>> >>
>> >> lxc-execute: Read-only file system - error unlinking
>> >> /usr/lib/x86_64-linux-gnu/lxc/dev/kmsg
>> >>
>> >> lxc-execute: failed to setup kmsg for 'test'
>> >> lxc-execute: Read-only file system - failed to create directory
>> >> '/usr/lib/x86_64-linux-gnu/lxc/lxc_putold'
>> >>
>> >> lxc-execute: Read-only file system - failed to create pivotdir
>> >> '/usr/lib/x86_64-linux-gnu/lxc/lxc_putold'
>> >> lxc-execute: failed to setup pivot root
>> >> lxc-execute: failed to set rootfs for 'test'
>> >> lxc-execute: failed to setup the container
>> >> lxc-execute: invalid sequence number 1. expected 2
>> >> lxc-execute: failed to spawn 'test'
>> >>
>> >>
>> >> Is it possible to have some insight on what the problem is here?
>> >> I am using Ubuntu 13.10, and my lxc is the one from the official repo
>> >> (1.0.0.alpha1).
>> >>
>> >> Thanks you in advance for your help,
>> >>
>> >> Antonin
>> >> _______________________________________________
>> >> lxc-users mailing list
>> >> lxc-users at lists.linuxcontainers.org
>> >> http://lists.linuxcontainers.org/listinfo/lxc-users
>> >
>> >
>> >
>> > _______________________________________________
>> > lxc-users mailing list
>> > lxc-users at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-users
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users