[Lxc-users] systemd inside LXC

[Serge Hallyn]

Quoting Michael H. Warfield (mhw at WittsEnd.com):
> On Mon, 2012-10-22 at 16:21 -0500, Serge Hallyn wrote:
> > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > On Mon, 2012-10-22 at 15:14 -0500, Serge Hallyn wrote:
> > > > Quoting Michael H. Warfield (mhw at wittsend.com):
> > > > > Serge,
> > > > > 
> > > > > On Mon, 2012-10-22 at 09:12 -0500, Serge Hallyn wrote:
> > > > > > Quoting Serge Hallyn (serge.hallyn at canonical.com):
> > > > > > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > > > > > On Sun, 2012-10-21 at 14:49 -0500, Serge Hallyn wrote:
> > > > > > > > > Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > > > > > > > > > Serge,
> > > > > > > > > > 
> > > > > > > > 
> > > > > > > > ...
> > > > > > > > 
> > > > > > > > > > Short of building a custom systemd, I don't know how to fix that problem
> > > > > > > > > > and I suspect this OP is going to run into this same thing (container
> > > > > > > > > > taking over host's console) and might explain some of what he's seeing.
> > > > > > > > > > Several of these look like they could cause problems (like /dev/pts in
> > > > > > > > > > there).  I've really reached an impasse at getting systemd (at least
> > > > > > > > > > Fedora 16 and 17) to work in a container without screwing up the host.
> > > > > > > > > > Prohibiting mounts entirely in the container might work but I suspect
> > > > > > > > > > (having read some systemd error messages) systemd is going to have some
> > > > > > > > > > serious heartburn there.
> > > > > > > > > > 
> > > > > > > > > > Thoughts?
> > > > > > > > > 
> > > > > > > > > IIRC, simply having apparmor(/selinux) refuse the mount of /dev by the
> > > > > > > > > container should work, i.e. systemd was not going to fail as a result.
> > > > > > > > 
> > > > > > > > Hopefully, you've seen the message from Kay Sievers cc'ed to this list
> > > > > > > > from my post to the systemd-devel list.  Looks like they have a
> > > > > > > > mechanism in place to do this...
> > > > > > > > 
> > > > > > > > http://www.freedesktop.org/wiki/Software/systemd/ContainerInterface
> > > > > > > 
> > > > > > > Saw the email, haven't yet read the page, thanks.
> > > > > 
> > > > > > So based on that page, what we do (set 'container=lxc') should already be
> > > > > > sufficient.
> > > > > 
> > > > > Thanks to the dude asking a libvirt-lxc question on the list, I was let
> > > > > to a page that let to a page that led to some discussion you were having
> > > > > back in March with Ramez Hanna on this very subject, "Re: [Lxc-users]
> > > > > f16 update"...
> > > > > 
> > > > > http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg03263.html
> > > > 
> > > > thanks, I knew we'd been over some of this, but couldn't find my logs of
> > > > it.
> > > > 
> > > > > This would look to be the kludge to make a workaround for this problem,
> > > > > I'm just not sure how to make it happen.  Given you already found the
> > > > > answer that the device for /dev has to be different than the device for
> > > > > the parent, what should we do.
> > > > > 
> > > > > I tried this in the config...
> > > > > 
> > > > > lxc.mount.entry=tmpfs /var/lib/lxc/private/Alcove/dev tmpfs defaults 0 0
> > > 
> > > > How about just a devtmpfs?  We actually now do this by default (as of very
> > > > recently) in ubuntu by adding
> > > 
> > > > devtmpfs        dev          devtmpfs defaults 0 0
> > > 
> > > NO!  That's the problem!  That leads to the container connecting to the
> > > hosts console and other devices and committing random acts of terrorism.
> 
> > No, it shouldn't, because lxc sets up the console after doing the mounts.
> 
> Maybe it shouldn't but that appears to be what is happening and even you
> remarked that maybe the problem was something doing a remount of /dev
> after entering the container...
> 
> I see your point though.  If you did that mount after LXC set up the
> console, then systemd wouldn't set it up and would drop into its more
> restricted mode.  That MIGHT help but you still have the entire dev
> space of the host exposed to the guest which is what you were talking
> about before wrt namespaces on devices.  It might help.  Would it be the
> answer?  Given that we've restricted access to those nodes in the
> config, maybe yes.  I'm just not so sure.  Will give it a shot though.
> 
> Strange, though, my earlier effort at tmpfs on dev had no effect.  Will
> give it a shot.

Thanks again for looking into this - I know just how frustrating it
can be!  :)

-serge