<div dir="ltr">I am just getting started with LXC. I'm using Ubuntu 12.04 (Precise). After a week of reading and experimenting, I have the beginnings of a working prototype and a handful of questions. :)<div><br></div>
<div>First, my use case. I'm running a number of customer-provided applications on a shared host. My only goal for LXC (at least for today) is to provide a read-only root filesystem for each application while the host itself has a read-write root filesystem (I have other security mechanisms in place, too, that are out of scope for this list). I do not want a fully contained virtual OS with its own init; I just want to contain the app. Like the recent thread "Sharing container rootfs", I do *not* want a separate copy of the rootfs for each application. I need to share the host's rootfs for every app, and I'll explicitly bind-mount in directories that need to be unique or read-write for each app (e.g. /tmp).</div>
<div><br></div><div style>On my prototype host, I have bind-mounted the host's root fs to /var/lib/lxc/cloud/rootfs with</div><div><br></div><div>mount --bind / /var/lib/lxc/cloud/rootfs</div><div>mount -o remount,ro /var/lib/lxc/cloud/rootfs</div>
<div style><br></div><div style>My prototype LXC config file is quite simple:<br></div><div style><br></div><div style>lxc.utsname = container</div><div style><div>lxc.rootfs = /var/lib/lxc/cloud/rootfs</div><div style># For each directory in the app container that needs to be read-write or unique:</div>
<div>lxc.mount.entry=/FOO /var/lib/lxc/cloud/rootfs/FOO none rw,bind 0 0<br></div><div><br></div><div style>I start the application with lxc-execute -n uniquename -f /var/lib/lxc/cloud/config -- [cmd]. This basically seems to work. The root fs within the container is read-only, and the directories I am explicitly bind over-mounting are read-write. Yay. I am also getting a pid namespace, I guess by default, which is fine. </div>
<div style><br></div><div style>Here are some questions:</div><div style><br></div><div style>* I'm currently sharing the host's /dev, which I know is a mistake. I can make my own inside rootfs and bind-mount it in. Any suggestions for a minimal device set to include? My goal is to prevent the application from escaping the container. I guess I can start with what the busybox template creates and add whatever we discover our customer apps need.</div>
<div style><br></div><div style>* lxc-attach does not work. This isn't that critical as I don't think I need it, I just want to make sure I'm not doing something wrong.</div><div style><br></div><div style><div>
host# lxc-attach -n test ps</div><div>lxc-attach: No such file or directory - failed to open '/proc/16808/ns/pid'</div><div>lxc-attach: failed to enter the namespace</div><div>host# ls /proc/16808/ns</div><div>ipc net uts</div>
<div><br></div><div style>Why does /proc/PID/ns/pid not exist? Also note that I did not create an ipc or net namespace (at least on purpose) but I do have those files.</div></div><div style><br></div><div style>* When I run "bash" as the command under lxc-execute, it is not connecting to the tty correctly (this isn't necessary a problem, since my customer apps are not bash and are not interactive, but it makes me think something is wrong that should be fixed). The first character typed results in "lxc-execute: Input/output error - failed to read". Nothing that I type is echoed correctly, though the output of all commands appears. Also, when the contained shell exits, lxc-execute gets stopped; it looks like "child process suspended because it generated output." Example, in which I type "ps<RET><Ctrl-D><RET>" at the "container#" prompt:</div>
<div style><br></div><div style><div><div>host# lxc-execute -n test2 -f /var/lib/lxc/cloud/config -- bash</div><div>container# lxc-execute: Input/output error - failed to read</div><div> PID TTY TIME CMD</div><div>
1 ? 00:00:00 lxc-init</div><div> 2 ? 00:00:00 bash</div><div> 12 ? 00:00:00 ps</div><div>container# exit<br></div><div><br></div><div>[2]+ Stopped lxc-execute -n test2 -f /var/lib/lxc/cloud/config -- bash</div>
<div>host# fg<br></div></div><div><div>lxc-execute -n test2 -f /var/lib/lxc/cloud/config -- bash</div><div>host#</div></div><div><br></div><div style>What is going on?</div></div><div style><br></div><div style>* Within my container, /etc/mtab is the host's mtab, so "mount" displays the host's mounts, while "cat /proc/mounts" display's the container's mounts (namespaced). I tried bind-mounting an empty, read-write file on top of /etc/mtab, but then it just stayed empty. What is supposed to update /etc/mtab? (For that matter, why isn't it just a symlink to /proc/mounts?)</div>
<div style><br></div><div style>I'm happy to hear other thoughts about things I'm missing.</div></div><div><br></div><div style>Thanks!</div><div style><br></div><div style>Barry</div><div style><br></div><div style>
PS: For the curious, the other security mechanism I'm experimenting with is setting and locking all "securebits" and setting the customer app's capability bounding set to be empty; see capabilities(7) for details (this turns out to be somewhat tricky to achieve, but is possible). Once done, even if the customer achieves root, it will not have any of root's superuser capabilities. However, it *will* have full access to files owned by uid 0; that's why I want the root filesystem to be read-only.</div>
<div><br></div><div>-- <br></div><div><div class="gmail_extra">Barry Jaspan<br><font size="1"><span style="color:rgb(102,102,102)">Senior Architect | </span><a style="color:rgb(102,102,102)" href="http://acquia.com" target="_blank">Acquia</a><br style="color:rgb(102,102,102)">
<a style="color:rgb(102,102,102)" href="mailto:barry.jaspan@acquia.com" target="_blank">barry.jaspan@acquia.com</a><span style="color:rgb(102,102,102)"> | (c) 617.905.2208 | (w) 781-313-8298</span><br style="color:rgb(102,102,102)">
<br style="color:rgb(102,102,102)"><font><a href="http://www.acquia.com/dev-cloud" target="_blank">Acquia Dev Cloud: You build killer websites. We do the rest.</a><a href="http://acquia.com/dev-cloud" target="_blank"></a></font></font><br>
<a href="http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing" target="_blank"><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:bold;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">Acquia</span><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline"> ranked #1 Software Vendor on the 2012 Inc 500</span></a><br>
<br>
</div></div></div>