[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie

[Michael H. Warfield]

On Tue, 2014-09-30 at 23:56 +0100, Chris wrote:
> On 30/09/14 19:28, Michael H. Warfield wrote:
> >> I haven't looked a whole lot into the premade containers, my gut feeling
> >> was that I didn't want to download a whole operating system from this
> >> project, and that I'd be a lot more comfortable taking distribution that
> >> I trust, and making the template manually. This way I know everything
> >> extra that's going into it.
> > Our templates are pretty barebones.  Very minimal.  You'll have to add
> > just about anything you would really want to make a useful container.
> I should definitely take a closer look sometime.
> >> It's running Debian Jessie. LXC 1.0.5-3 from package management. And
> >> systemd 208-8 also from package management.
> > OK... THAT explains a LOT!  That systemd option is why you're running
> > into this problem and you're about to have far worse.
> >
> >> I took a config from an existing container and modified it for what I
> >> thought would work for an unprivileged container. I've attached the
> >> config for osmium. I've also attached the latest trace output from the
> >> lxc-start, as I've fixed a few slight errors in the config since then.
> > You're going to have to make some additional changes...  Make sure you
> > add "lxc.kmsg = 0" to your container or systemd.journald is going to eat
> > your CPU time for lunch (and be sure to flush
> > your /dev/.lxc/user/osmium* directory).  There's also some adjustments
> > that need to be made for mgetty consoles and such.  You also need to
> > link the shutdown unit to the SIGPWR service to allow lxc to shut the
> > container down gracefully.  You might take a look at the Oracle or
> > Fedora templates for some guidance there.
> Will definitely come back to this once it starts up, thank you for the 
> advice.
> >> osmium at cadmium:~$ find /dev/.lxc/user -ls
> >>     9668    0 drwxrwxrwt   3 root     root           60 Sep 30 15:38
> >> /dev/.lxc/user
> >>    11109    0 drwxr-xr-x   3 427680   427680         60 Sep 30 15:38
> >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d
> >>    11110    0 drwxr-xr-x   2 427680   427680         40 Sep 30 15:38
> >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts
> > Bingo!
> >
> > Ok...  So it appears that lxc-start did manage to create your dev
> > directory properly under the host /dev/.lxc/user.
> >
> > Now I see the real problem...
> >
> > The same code that creates that directory creates the symlink
> > in /home/osmium/.local/share/lxc/osmium.  But, the /dev/ directory is
> > owned by "427680:427680" while the directory containing the symlink is
> > own by "osmium:osmium" and you then have a permission denied because
> > 427680:427680 doesn't have write permissions
> > to /home/osmium/.local/share/lxc/osmium.
> >
> > That's a (the!) problem.  I'm just not sure if chown/chgrp is the
> > correct answer or if you need to add some group membership and add group
> > write permissions with appropriate host auth secondary groups.  Either
> > way, it's that permission problem that biting you in the rear end.
> >
> OK, yes. This was that problem. Fixing it has progressed startup a 
> little further. It didn't like the lxc.mount.entry for devpts, so I 
> threw that out for the time being also. Now it's still stuck at 
> 'populating dev' though. I've attached the latest trace in case you help 
> me again.
> 
> osmium at cadmium:~$ lxc-start -n osmium -l trace -o /tmp/xxx7
> lxc-start: Operation not permitted - Error creating null

Ouch.  Looks like it's attempting to do a mknod which is not allowed and
can not be allowed as it's prohibited in the kernel to any non-priv
users.  That's a security issue that can not be changed.  That's going
to be a problem for any autodev containers running under non-priv users.
Serge and/or Stéphane will have to respond to that one.

That was actually a case I was considering when I was discussing the
whole devtmpfs thing with Serge back at LinuxPlumbers last year.
Non-priv users can not create devices (mknod), period, end of
discussion.  But they might be able to create hard links to them within
devtmpfs or do bind mounts, which might work under the current setup.
That was the intent at least.  I don't recall the hard link device code
being done (though we do bind mounts on some devices) so I'm not sure
what's transpiring at this point in the code.

I'm also not sure about your pts comment above.  That's a bind mount as
well (unless you had something else in there).  That should also work.
This may be pointing up some underlying problem with getting systemd and
autodev working in a non-priv environment.

> lxc-start: failed to populate /dev in the container
> lxc-start: failed to setup the container
> lxc-start: invalid sequence number 1. expected 2
> lxc-start: failed to spawn 'osmium'
> lxc-start: The container failed to start.
> lxc-start: Additional information can be obtained by setting the 
> --logfile and --log-priority options

> Thanks,
> Chris

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 978-7061 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140930/31a7439b/attachment.sig>