[lxc-devel] [Lxc-users] [systemd-devel] Unable to run systemd in an LXC / cgroup container.

[Michael H. Warfield]

On Sun, 2012-10-28 at 23:02 +0100, Serge Hallyn wrote:
> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > On Sun, 2012-10-28 at 18:52 +0100, Serge Hallyn wrote:

:

> > > I've got a rather minimal patch (appended below) to add the support for
> > > mounting and populating a minimal /dev working.  (A few hours were wasted
> > > due to my not knowing that upstart was going to issue mounted-dev even though
> > > /dev was mounted before upstart started - and the mounted-dev hook deletes
> > > and recreates all consoles.  GAH)
> > 
> > I am happy to report that, after manually editing my git head branch to

> Sorry, it was against the ubuntu quantal package.  I've been in the air
> without onboard wifi, so working with what i had at hand.

Oh, I figured it was a package mismatch.  Wasn't too terribly difficult
to patch those hunks in and kick out a diff against git.

> > patch in the failed hunks, I was able to build it and test it and my
> > Fedora 17 systemd based container fired right up after adding the
> > lxc.autodev = 1 parameter to the config file.  Yeah!!!!
> > 
> > I did run into one gotcha, but one I can live with.  I had been bind
> > mounting the private root file system to another directory and then
> > using that as the rootfs like this:
> > 
> > -- 
> > lxc.rootfs = /srv/lxc/rootfs
> > lxc.mount.entry=/srv/lxc/private/Alcove /srv/lxc/rootfs none bind.shared 0 0
> > lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0
> > 
> > lxc.autodev = 1
> > -- 
> > 
> > This did not work and I got the startup error that it can not mount
> > to /dev because it doesn't exist.

> Hm, yeah.  If you do need to play a game like this, you might be best
> off using a pre-mount hook for that.

Yeah, I don't think I "need to play a game like this" anymore.  I'd have
to go back through some old old E-Mails to see why I did that before.  I
seem to recall we were playing with all sorts of bind mount options for
some PRIVATE thing or another.  It may not be necessary at all any
longer.  IAC, it's minor to switch it back.  I seem to recall switching
back and forth using bind mounts several times back when that got done
that way.

I may play with the pre-mount hook just for giggles and see how that
might work as well.  Any idea why I was experiencing the problem with
the mount hook when trying to populate /dev?  I know it wouldn't have
worked because of the /dev/pts mount but I have more heartburn in that
it looks like it ran too early and the mount on /dev had not even taken
place at that time.

> > I believe I can see why...  You're doing the autodev populate prior to
> > any of the mounts being performed, so that "private" root file system is
> > not bound to the directory at that time.
> > 
> > Drop that bind mount for the rootfs and this then worked like a charm:
> > 
> > -- 
> > lxc.rootfs = /srv/lxc/private/Alcove
> > lxc.mount.entry=/home/shared /srv/lxc/private/Alcove/srv/shared none ro,bind 0 0
> > 
> > lxc.autodev = 1
> > -- 
> > 
> > I think that rootfs directory bind was an effort to more fully match the
> > OpenVZ behavior but also trying to deal with some of the read-only
> > problems were where having in the past with shutdowns.  If it won't
> > work, it won't work and I won't miss it.
> > 
> > I did see some errors setting up that dev...
> > 
> > -- 
> > [root at forest mhw]# lxc-start -n Alcove
> > lxc-start: No such file or directory - failed to mount '/dev/pts/59'->'/usr/lib64/lxc/rootfs/dev/tty1'
> > lxc-start: No such file or directory - failed to mount '/dev/pts/60'->'/usr/lib64/lxc/rootfs/dev/tty2'
> > lxc-start: No such file or directory - failed to mount '/dev/pts/61'->'/usr/lib64/lxc/rootfs/dev/tty3'
> > lxc-start: No such file or directory - failed to mount '/dev/pts/62'->'/usr/lib64/lxc/rootfs/dev/tty4'
> > lxc-start: No such file or directory - failed to mount '/dev/pts/63'->'/usr/lib64/lxc/rootfs/dev/tty5'
> > lxc-start: No such file or directory - failed to mount '/dev/pts/64'->'/usr/lib64/lxc/rootfs/dev/tty6'
> > systemd 44 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP; fedora)
> > 
> > Welcome to Fedora 17 (Beefy Miracle)!
> > 
> > -- 
> > 
> > Not sure what that's all about but, since systemd isn't going to start
> > getty's on the tty? interfaces anyways, it probably doesn't make much
> > difference.

> Oh, I see.  Yeah, in the !lxc.ttydir case, when we created our own /dev
> we should create the tty files.  I need to fix that.

Cool.  Once again...  Looks like we got some real progress here with
this one.  I've still got more testing to do, undoing some of my changes
in the container itself and making sure it all still works.

Also looks like I can stop and restart one of these containers now
without the hung cgroup directory.  I suspected it was some stray
devices behind that.  This is good.

> Of course in your case since systemd isn't going to start getty's on
> them, you should not have the lxc.tty = 6 in your container config,
> which it looks like you still do?

Yeah.  I was taking it one step at a time.  I wish they WOULD fire up
some getty's on those tty's since that basically makes lxc-console kinda
useless and the one on /dev/console is kinda useless in disconnected
mode with the console redirected to a file.  Maybe we need some what for
lxc-console to be able to grab that?  I'll have to look deeper at
systemd and figure out if it can be parameterized or conditionalized in
some way.  ITMT, I probably will just turn them off.

Many thanks!

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20121028/d93e415b/attachment.pgp>