[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie

[Michael H. Warfield]

On Tue, 2014-09-30 at 17:45 +0100, Chris wrote:
> On 30/09/14 16:47, Michael H. Warfield wrote:
> > On Tue, 2014-09-30 at 15:46 +0100, Chris wrote:
> >> On 29/09/14 21:46, Serge Hallyn wrote:
> >>> Hm, sorry, not looking deeper right now, but :
> >>>
> >>>>         lxc-start 1411807327.953 ERROR    lxc_conf - Permission denied - WARNING: Failed to create symlink '/home/osmium/.local/share/lxc/osmium/rootfs.dev'->'/dev/.lxc/user/osmium.3c68b3f0c5eeec7d'
> >>> Something will need to set that up.  I can't recall offhand
> >>> what is supposed to do that.  Michael (cc:d), is that done
> >>> through the init script?
> >>>
> >>> -serge
> >>>
> >>>
> >> That might make sense, as I created this container through
> >> debootstrapping the filesystem into
> >> /home/osmium/.local/share/lxc/osmium/rootfs and then chown/grping all
> >> the files to the appropriate users in this user's subuid/gid range...
> >> pasted below in case anyone finds it useful. Please let me know if there
> >> are further steps required to make this template/container valid.
> > You created this with debootstrap?  So it's an Ubuntu or Debian
> > container?  Why not use the appropriate lxc-create template?  They do a
> > lot of things that you are unlikely to have done.  Since you're creating
> > a container for an unprivileged user, you should probably have used the
> > download template, as the live templates are generally for privileged
> > users only.

> I haven't looked a whole lot into the premade containers, my gut feeling 
> was that I didn't want to download a whole operating system from this 
> project, and that I'd be a lot more comfortable taking distribution that 
> I trust, and making the template manually. This way I know everything 
> extra that's going into it.

Our templates are pretty barebones.  Very minimal.  You'll have to add
just about anything you would really want to make a useful container.

> > That error is generated out of the code, which I authored, that sets up
> > the autodev device areas and mounts that systemd mandates (but can still
> > be used by anyone).  But, if this is Debian or Ubuntu, what version did
> > you attempt to install?  Unless you're loading a test version, you
> > shouldn't be getting systemd as your default init system manager (yet).
> > If you have not explicitly set lxc.autodev = 1 in the config file and
> > lxc-start does not detect systemd as the init system, you should not
> > have ventured into that code at all.  I'm really baffled how you got in
> > a situation where you used debootstrap and yet the code is running into
> > the systemd autodev logic, something I would not have expected for
> > Ubuntu or Debian just yet (and I don't think those templates are
> > prepared to set up just yet).

> It's running Debian Jessie. LXC 1.0.5-3 from package management. And 
> systemd 208-8 also from package management.

OK... THAT explains a LOT!  That systemd option is why you're running
into this problem and you're about to have far worse.

> > Next question...  How did you create your configuration file?  That
> > error message is telling me that either you had lxc.autodev == 1 in the
> > configuration file OR you're running systemd as your init system
> > manager.  Neither of those should be a particular problem (well, systemd
> > might if you haven't properly configured certain aspects of the unit
> > files are startup - but you aren't getting that far) but it's just not
> > clear how you got where you got doing what you did.

> I took a config from an existing container and modified it for what I 
> thought would work for an unprivileged container. I've attached the 
> config for osmium. I've also attached the latest trace output from the 
> lxc-start, as I've fixed a few slight errors in the config since then.

You're going to have to make some additional changes...  Make sure you
add "lxc.kmsg = 0" to your container or systemd.journald is going to eat
your CPU time for lunch (and be sure to flush
your /dev/.lxc/user/osmium* directory).  There's also some adjustments
that need to be made for mgetty consoles and such.  You also need to
link the shutdown unit to the SIGPWR service to allow lxc to shut the
container down gracefully.  You might take a look at the Oracle or
Fedora templates for some guidance there.

> > What are the permissions on /home/osmium/.local/share/lxc/osmium ?  For
> > some reason, lxc-start does not have permission to create a symlink in
> > that directory (or maybe does not have rx read/search permission to all
> > of its parent directories in the path).  That's a short-cut link back to
> > the hash indexed dev directory under /dev/.lxc/user (for unpriv users)
> > for the container /dev.  Creating that symlink depends only on the
> > permissions in the path to the directory and the directory itself.
> >
> > Regards,
> > Mike
> >
> >
> osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium
> drwxr-xr-x 3 osmium osmium 4096 Sep 30 15:38 
> /home/osmium/.local/share/lxc/osmium
> osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium/rootfs/
> drwxr-xr-x 21 427680 427680 4096 Sep 14 15:56 
> /home/osmium/.local/share/lxc/osmium/rootfs/
> osmium at cadmium:~$ ls -ld /home/osmium/.local/share/lxc/osmium/rootfs/dev
> drwxr-xr-x 4 427680 427680 4096 Sep 14 15:56 
> /home/osmium/.local/share/lxc/osmium/rootfs/dev

> osmium at cadmium:~$ grep osmium /etc/sub[ug]id
> /etc/subgid:osmium:427680:65536
> /etc/subuid:osmium:427680:65536

> osmium at cadmium:~$ find /dev/.lxc/user -ls
>    9668    0 drwxrwxrwt   3 root     root           60 Sep 30 15:38 
> /dev/.lxc/user
>   11109    0 drwxr-xr-x   3 427680   427680         60 Sep 30 15:38 
> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d
>   11110    0 drwxr-xr-x   2 427680   427680         40 Sep 30 15:38 
> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts

Bingo!

Ok...  So it appears that lxc-start did manage to create your dev
directory properly under the host /dev/.lxc/user.

Now I see the real problem...

The same code that creates that directory creates the symlink
in /home/osmium/.local/share/lxc/osmium.  But, the /dev/ directory is
owned by "427680:427680" while the directory containing the symlink is
own by "osmium:osmium" and you then have a permission denied because
427680:427680 doesn't have write permissions
to /home/osmium/.local/share/lxc/osmium.

That's a (the!) problem.  I'm just not sure if chown/chgrp is the
correct answer or if you need to add some group membership and add group
write permissions with appropriate host auth secondary groups.  Either
way, it's that permission problem that biting you in the rear end.

> Thanks,
> Chris

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 978-7061 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140930/11109453/attachment-0001.sig>