[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie

[Michael H. Warfield]

On Tue, 2014-09-30 at 19:39 -0400, Michael H. Warfield wrote:
> On Tue, 2014-09-30 at 23:56 +0100, Chris wrote:
> > On 30/09/14 19:28, Michael H. Warfield wrote:
> > >> I haven't looked a whole lot into the premade containers, my gut feeling
> > >> was that I didn't want to download a whole operating system from this
> > >> project, and that I'd be a lot more comfortable taking distribution that
> > >> I trust, and making the template manually. This way I know everything
> > >> extra that's going into it.
> > > Our templates are pretty barebones.  Very minimal.  You'll have to add
> > > just about anything you would really want to make a useful container.
> > I should definitely take a closer look sometime.
> > >> It's running Debian Jessie. LXC 1.0.5-3 from package management. And
> > >> systemd 208-8 also from package management.
> > > OK... THAT explains a LOT!  That systemd option is why you're running
> > > into this problem and you're about to have far worse.
> > >
> > >> I took a config from an existing container and modified it for what I
> > >> thought would work for an unprivileged container. I've attached the
> > >> config for osmium. I've also attached the latest trace output from the
> > >> lxc-start, as I've fixed a few slight errors in the config since then.
> > > You're going to have to make some additional changes...  Make sure you
> > > add "lxc.kmsg = 0" to your container or systemd.journald is going to eat
> > > your CPU time for lunch (and be sure to flush
> > > your /dev/.lxc/user/osmium* directory).  There's also some adjustments
> > > that need to be made for mgetty consoles and such.  You also need to
> > > link the shutdown unit to the SIGPWR service to allow lxc to shut the
> > > container down gracefully.  You might take a look at the Oracle or
> > > Fedora templates for some guidance there.
> > Will definitely come back to this once it starts up, thank you for the 
> > advice.
> > >> osmium at cadmium:~$ find /dev/.lxc/user -ls
> > >>     9668    0 drwxrwxrwt   3 root     root           60 Sep 30 15:38
> > >> /dev/.lxc/user
> > >>    11109    0 drwxr-xr-x   3 427680   427680         60 Sep 30 15:38
> > >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d
> > >>    11110    0 drwxr-xr-x   2 427680   427680         40 Sep 30 15:38
> > >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts
> > > Bingo!
> > >
> > > Ok...  So it appears that lxc-start did manage to create your dev
> > > directory properly under the host /dev/.lxc/user.
> > >
> > > Now I see the real problem...
> > >
> > > The same code that creates that directory creates the symlink
> > > in /home/osmium/.local/share/lxc/osmium.  But, the /dev/ directory is
> > > owned by "427680:427680" while the directory containing the symlink is
> > > own by "osmium:osmium" and you then have a permission denied because
> > > 427680:427680 doesn't have write permissions
> > > to /home/osmium/.local/share/lxc/osmium.
> > >
> > > That's a (the!) problem.  I'm just not sure if chown/chgrp is the
> > > correct answer or if you need to add some group membership and add group
> > > write permissions with appropriate host auth secondary groups.  Either
> > > way, it's that permission problem that biting you in the rear end.
> > >
> > OK, yes. This was that problem. Fixing it has progressed startup a 
> > little further. It didn't like the lxc.mount.entry for devpts, so I 
> > threw that out for the time being also. Now it's still stuck at 
> > 'populating dev' though. I've attached the latest trace in case you help 
> > me again.
> > 
> > osmium at cadmium:~$ lxc-start -n osmium -l trace -o /tmp/xxx7
> > lxc-start: Operation not permitted - Error creating null

> Ouch.  Looks like it's attempting to do a mknod which is not allowed and
> can not be allowed as it's prohibited in the kernel to any non-priv
> users.  That's a security issue that can not be changed.  That's going
> to be a problem for any autodev containers running under non-priv users.
> Serge and/or Stéphane will have to respond to that one.

> That was actually a case I was considering when I was discussing the
> whole devtmpfs thing with Serge back at LinuxPlumbers last year.
> Non-priv users can not create devices (mknod), period, end of
> discussion.  But they might be able to create hard links to them within
> devtmpfs or do bind mounts, which might work under the current setup.
> That was the intent at least.  I don't recall the hard link device code
> being done (though we do bind mounts on some devices) so I'm not sure
> what's transpiring at this point in the code.

Maybe I did test that out last year.  IAC...  Just retested it.  Damn
it, oh yeah...  Hardlinks to device nodes in devtmpfs are prohibited to
non-priv users even when permissions would permit.  That option is also
non viable.  Nice idea.  Guess I know why none of us pursued it now.

> I'm also not sure about your pts comment above.  That's a bind mount as
> well (unless you had something else in there).  That should also work.
> This may be pointing up some underlying problem with getting systemd and
> autodev working in a non-priv environment.
> 
> > lxc-start: failed to populate /dev in the container
> > lxc-start: failed to setup the container
> > lxc-start: invalid sequence number 1. expected 2
> > lxc-start: failed to spawn 'osmium'
> > lxc-start: The container failed to start.
> > lxc-start: Additional information can be obtained by setting the 
> > --logfile and --log-priority options
> 
> > Thanks,
> > Chris
> 
> Regards,
> Mike

-- 
Michael H. Warfield (AI4NB) | (770) 978-7061 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140930/33482fcf/attachment.sig>