[Lxc-users] Containing a user-space application

[Barry Jaspan]

I am just getting started with LXC. I'm using Ubuntu 12.04 (Precise). After
a week of reading and experimenting, I have the beginnings of a working
prototype and a handful of questions. :)

First, my use case. I'm running a number of customer-provided applications
on a shared host. My only goal for LXC (at least for today) is to provide a
read-only root filesystem for each application while the host itself has a
read-write root filesystem (I have other security mechanisms in place, too,
that are out of scope for this list). I do not want a fully contained
virtual OS with its own init; I just want to contain the app. Like the
recent thread "Sharing container rootfs", I do *not* want a separate copy
of the rootfs for each application. I need to share the host's rootfs for
every app, and I'll explicitly bind-mount in directories that need to be
unique or read-write for each app (e.g. /tmp).

On my prototype host, I have bind-mounted the host's root fs to
/var/lib/lxc/cloud/rootfs with

mount --bind / /var/lib/lxc/cloud/rootfs
mount -o remount,ro /var/lib/lxc/cloud/rootfs

My prototype LXC config file is quite simple:

lxc.utsname = container
lxc.rootfs = /var/lib/lxc/cloud/rootfs
# For each directory in the app container that needs to be read-write or
unique:
lxc.mount.entry=/FOO /var/lib/lxc/cloud/rootfs/FOO none rw,bind 0 0

I start the application with lxc-execute -n uniquename -f
/var/lib/lxc/cloud/config -- [cmd]. This basically seems to work. The root
fs within the container is read-only, and the directories I am explicitly
bind over-mounting are read-write. Yay. I am also getting a pid namespace,
I guess by default, which is fine.

Here are some questions:

* I'm currently sharing the host's /dev, which I know is a mistake. I can
make my own inside rootfs and bind-mount it in. Any suggestions for a
minimal device set to include? My goal is to prevent the application from
escaping the container. I guess I can start with what the busybox template
creates and add whatever we discover our customer apps need.

* lxc-attach does not work. This isn't that critical as I don't think I
need it, I just want to make sure I'm not doing something wrong.

host# lxc-attach -n test ps
lxc-attach: No such file or directory - failed to open '/proc/16808/ns/pid'
lxc-attach: failed to enter the namespace
host# ls /proc/16808/ns
ipc  net  uts

Why does /proc/PID/ns/pid not exist? Also note that I did not create an ipc
or net namespace (at least on purpose) but I do have those files.

* When I run "bash" as the command under lxc-execute, it is not connecting
to the tty correctly (this isn't necessary a problem, since my customer
apps are not bash and are not interactive, but it makes me think something
is wrong that should be fixed). The first character typed results in
"lxc-execute: Input/output error - failed to read". Nothing that I type is
echoed correctly, though the output of all commands appears. Also, when the
contained shell exits, lxc-execute gets stopped; it looks like "child
process suspended because it generated output." Example, in which I type
"ps<RET><Ctrl-D><RET>" at the "container#" prompt:

host# lxc-execute -n test2 -f /var/lib/lxc/cloud/config -- bash
container# lxc-execute: Input/output error - failed to read
  PID TTY          TIME CMD
    1 ?        00:00:00 lxc-init
    2 ?        00:00:00 bash
   12 ?        00:00:00 ps
container# exit

[2]+  Stopped                 lxc-execute -n test2 -f
/var/lib/lxc/cloud/config -- bash
host# fg
lxc-execute -n test2 -f /var/lib/lxc/cloud/config -- bash
host#

What is going on?

* Within my container, /etc/mtab is the host's mtab, so "mount" displays
the host's mounts, while "cat /proc/mounts" display's the container's
mounts (namespaced). I tried bind-mounting an empty, read-write file on top
of /etc/mtab, but then it just stayed empty. What is supposed to update
/etc/mtab? (For that matter, why isn't it just a symlink to /proc/mounts?)

I'm happy to hear other thoughts about things I'm missing.

Thanks!

Barry

PS: For the curious, the other security mechanism I'm experimenting with is
setting and locking all "securebits" and setting the customer app's
capability bounding set to be empty; see capabilities(7) for details (this
turns out to be somewhat tricky to achieve, but is possible). Once done,
even if the customer achieves root, it will not have any of root's
superuser capabilities. However, it *will* have full access to files owned
by uid 0; that's why I want the root filesystem to be read-only.

-- 
Barry Jaspan
Senior Architect | Acquia <http://acquia.com>
barry.jaspan at acquia.com | (c) 617.905.2208 | (w) 781-313-8298

Acquia Dev Cloud: You build killer websites. We do the
rest.<http://www.acquia.com/dev-cloud><http://acquia.com/dev-cloud>
Acquia ranked #1 Software Vendor on the 2012 Inc
500<http://www.acquia.com/about-us/newsroom/press-releases/inc-magazine-unveils-31st-annual-list-america-s-fastest-growing>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20130610/214ca131/attachment.html>