[Lxc-users] systemd inside LXC

Sat Oct 20 23:41:59 UTC 2012

Serge,

I'm going to top post here simply because this is going to go off in a
different direction and bringing in an old thread but it is related...

Back on February 14 you responded to a message about Fedora 16 in a
container, which is something I've been trying to do and I had run into
that posters problems.  You responded with this:

Subject: Re: [Lxc-users] fedora 16 under lxc

On Tue, 2012-02-14 at 09:23 -0600, Serge Hallyn wrote:
> Quoting Ramez Hanna (rhanna at informatiq.org):

> > now all my efforts have not succeedd to get getty on tty1 to start
> > unmasking udev did something different
> > it created all the /dev devices
> > and made getty start but it started on the hosts's tty not on the container's
> > could someone shed some light here?
> 
> Blind guess:
> 
> lxc-start creates some ptys and bind mounts them onto the guest's
> /dev/{console,tty{1,2,3,4}}.  It sounds like fedora's init is mounting
> over the /dev set up by lxc causing a new /dev/tty to be created as
> chardev 4:{1-4}.  Devices namespaces would help this.  We're hoping to
> discuss design for those at next UDS, but those will come after user
> namespaces.  In the mean time, you'll need to make sure that the guest
> does not mount over /dev, and does not remount /dev/pts.
> 
> -serge

That got me thinking and started into looking deeper into systemd, which
Fedora 16 and above uses and why it may be related here.  I've made
Fedora 16 work in the past by installing upstart and disabling systemd
but that really becomes impractical in Fedora 17 because they're
including so few of the compatibility scripts.  Yes, you are right, the
Fedora's init (systemd) is mounting something on /dev like this:

devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1784160k,nr_inodes=446040,mode=755)

This is very bad for the reasons you pointed out in Feb.  Looking at the
source code for systemd, this is hard coded into the binary and is not
configurable.

systemd-37/src/mount-setup.c:
-- 
/* The first three entries we might need before SELinux is up. The
 * other ones we can delay until SELinux is loaded. */
#define N_EARLY_MOUNT 3

static const MountPoint mount_table[] = {
        { "proc",     "/proc",                  "proc",     NULL,                MS_NOSUID|MS_NOEXEC|MS_NODEV, true },
        { "sysfs",    "/sys",                   "sysfs",    NULL,                MS_NOSUID|MS_NOEXEC|MS_NODEV, true },
        { "devtmpfs", "/dev",                   "devtmpfs", "mode=755",          MS_NOSUID,                    true },
        { "tmpfs",    "/dev/shm",               "tmpfs",    "mode=1777",         MS_NOSUID|MS_NODEV,           true },
        { "devpts",   "/dev/pts",               "devpts",   "mode=620,gid=" STRINGIFY(TTY_GID), MS_NOSUID|MS_NOEXEC, false },
        { "tmpfs",    "/run",                   "tmpfs",    "mode=755",          MS_NOSUID|MS_NODEV, true },
        { "tmpfs",    "/sys/fs/cgroup",         "tmpfs",    "mode=755",          MS_NOSUID|MS_NOEXEC|MS_NODEV, false },
        { "cgroup",   "/sys/fs/cgroup/systemd", "cgroup",   "none,name=systemd", MS_NOSUID|MS_NOEXEC|MS_NODEV, false },
};
-- 

Short of building a custom systemd, I don't know how to fix that problem
and I suspect this OP is going to run into this same thing (container
taking over host's console) and might explain some of what he's seeing.
Several of these look like they could cause problems (like /dev/pts in
there).  I've really reached an impasse at getting systemd (at least
Fedora 16 and 17) to work in a container without screwing up the host.
Prohibiting mounts entirely in the container might work but I suspect
(having read some systemd error messages) systemd is going to have some
serious heartburn there.

Thoughts?

Regards,
Mike

On Fri, 2012-10-19 at 10:51 -0500, Serge Hallyn wrote:
> Quoting John (lxc at jelmail.com):
> > Hello, I'm in the middle of a migration from init to systemd. I've 
> > completed the transition of my host environment and my 6 existing 
> > containers continue to work as expected (they all use sysvinit 
> > internally). I've started work on a systemd container and am getting 
> > some odd effects.
> > 
> > First off, if I use systemd-nspawn to start the container, it starts 
> > fine. I can log in and halt it and all goes as expected. If, however I 
> > use lxc-start, it clobbers my desktop, which is running in another 
> > container.
> > 
> > So I have 2 problems: (a) the container does not boot and (b) it manages 
> > to effect changes in another container.
> > 
> > I've been searching the 'net for most of this morning looking for 
> > information on using systemd inside a container.
> > 
> > I'm using Arch Linux (3.6.2-1-ARCH) with LXC 0.8.0-rc2. Arch now uses 
> > systemd by default.
> > 
> > To try to test this, I created a basic container and this exhibits the 
> > same problems:
> > 
> > $ mkarchroot test base
> > 
> > Starting with systemd-nspawn works fine:
> > $  systemd-nspawn -D test/ /sbin/init
> > 
> > Starting with LXC does not:
> > $ lxc-create -n test -f test.conf
> > $ lxc-start -n test
> > 
> > The file test.conf contains these two lines:
> > 
> > lxc.utsname = test2
> > lxc.rootfs = /srv/lxc/test
> 
> Add:
> 
> lxc.network.type = empty
> 
> If you don't have any lxc.network.type sections, then the container
> shares network with the host, and so the container talks to the host's
> systemd.  (same with upstart)
> 
> > When I start the container in LXC, all that happens is that my X session 
> > dies (this is running in another container). The X session re-starts but 
> > the keyboard does not work. I have to connect using another machine to 
> > kill the test container and re-start my desktop container. I can't see 
> > anything starting inside the test container.
> > 
> > I'd be grateful for any help and/or pointers in the right direction so I 
> > can complete this transition to systemd.
> > 
> > Many thanks,
> > John
> > 
> > 
> > 
> > ------------------------------------------------------------------------------
> > Everyone hates slow websites. So do we.
> > Make your web apps faster with AppDynamics
> > Download AppDynamics Lite for free today:
> > http://p.sf.net/sfu/appdyn_sfd2d_oct
> > _______________________________________________
> > Lxc-users mailing list
> > Lxc-users at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/lxc-users
> 
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_sfd2d_oct
> _______________________________________________
> Lxc-users mailing list
> Lxc-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users
> 

-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20121020/9506b7ae/attachment.pgp>