[lxc-users] Unable to Start Unprivileged Containers on Debian / Jessie
Michael H. Warfield
mhw at WittsEnd.com
Wed Oct 1 14:19:28 UTC 2014
On Wed, 2014-10-01 at 13:53 +0000, Serge Hallyn wrote:
> Quoting Michael H. Warfield (mhw at WittsEnd.com):
> > On Tue, 2014-09-30 at 19:39 -0400, Michael H. Warfield wrote:
> > > On Tue, 2014-09-30 at 23:56 +0100, Chris wrote:
> > > > On 30/09/14 19:28, Michael H. Warfield wrote:
> > > > >> I haven't looked a whole lot into the premade containers, my gut feeling
> > > > >> was that I didn't want to download a whole operating system from this
> > > > >> project, and that I'd be a lot more comfortable taking distribution that
> > > > >> I trust, and making the template manually. This way I know everything
> > > > >> extra that's going into it.
> > > > > Our templates are pretty barebones. Very minimal. You'll have to add
> > > > > just about anything you would really want to make a useful container.
> > > > I should definitely take a closer look sometime.
> > > > >> It's running Debian Jessie. LXC 1.0.5-3 from package management. And
> > > > >> systemd 208-8 also from package management.
> > > > > OK... THAT explains a LOT! That systemd option is why you're running
> > > > > into this problem and you're about to have far worse.
> > > > >
> > > > >> I took a config from an existing container and modified it for what I
> > > > >> thought would work for an unprivileged container. I've attached the
> > > > >> config for osmium. I've also attached the latest trace output from the
> > > > >> lxc-start, as I've fixed a few slight errors in the config since then.
> > > > > You're going to have to make some additional changes... Make sure you
> > > > > add "lxc.kmsg = 0" to your container or systemd.journald is going to eat
> > > > > your CPU time for lunch (and be sure to flush
> > > > > your /dev/.lxc/user/osmium* directory). There's also some adjustments
> > > > > that need to be made for mgetty consoles and such. You also need to
> > > > > link the shutdown unit to the SIGPWR service to allow lxc to shut the
> > > > > container down gracefully. You might take a look at the Oracle or
> > > > > Fedora templates for some guidance there.
> > > > Will definitely come back to this once it starts up, thank you for the
> > > > advice.
> > > > >> osmium at cadmium:~$ find /dev/.lxc/user -ls
> > > > >> 9668 0 drwxrwxrwt 3 root root 60 Sep 30 15:38
> > > > >> /dev/.lxc/user
> > > > >> 11109 0 drwxr-xr-x 3 427680 427680 60 Sep 30 15:38
> > > > >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d
> > > > >> 11110 0 drwxr-xr-x 2 427680 427680 40 Sep 30 15:38
> > > > >> /dev/.lxc/user/osmium.3c68b3f0c5eeec7d/pts
> > > > > Bingo!
> > > > >
> > > > > Ok... So it appears that lxc-start did manage to create your dev
> > > > > directory properly under the host /dev/.lxc/user.
> > > > >
> > > > > Now I see the real problem...
> > > > >
> > > > > The same code that creates that directory creates the symlink
> > > > > in /home/osmium/.local/share/lxc/osmium. But, the /dev/ directory is
> > > > > owned by "427680:427680" while the directory containing the symlink is
> > > > > own by "osmium:osmium" and you then have a permission denied because
> > > > > 427680:427680 doesn't have write permissions
> > > > > to /home/osmium/.local/share/lxc/osmium.
> > > > >
> > > > > That's a (the!) problem. I'm just not sure if chown/chgrp is the
> > > > > correct answer or if you need to add some group membership and add group
> > > > > write permissions with appropriate host auth secondary groups. Either
> > > > > way, it's that permission problem that biting you in the rear end.
> > > > >
> > > > OK, yes. This was that problem. Fixing it has progressed startup a
> > > > little further. It didn't like the lxc.mount.entry for devpts, so I
> > > > threw that out for the time being also. Now it's still stuck at
> > > > 'populating dev' though. I've attached the latest trace in case you help
> > > > me again.
> > > >
> > > > osmium at cadmium:~$ lxc-start -n osmium -l trace -o /tmp/xxx7
> > > > lxc-start: Operation not permitted - Error creating null
> >
> > > Ouch. Looks like it's attempting to do a mknod which is not allowed and
> > > can not be allowed as it's prohibited in the kernel to any non-priv
> > > users. That's a security issue that can not be changed. That's going
> > > to be a problem for any autodev containers running under non-priv users.
> > > Serge and/or Stéphane will have to respond to that one.
> >
> > > That was actually a case I was considering when I was discussing the
> > > whole devtmpfs thing with Serge back at LinuxPlumbers last year.
> > > Non-priv users can not create devices (mknod), period, end of
> > > discussion. But they might be able to create hard links to them within
> > > devtmpfs or do bind mounts, which might work under the current setup.
> > > That was the intent at least. I don't recall the hard link device code
> > > being done (though we do bind mounts on some devices) so I'm not sure
> > > what's transpiring at this point in the code.
> >
> > Maybe I did test that out last year. IAC... Just retested it. Damn
> > it, oh yeah... Hardlinks to device nodes in devtmpfs are prohibited to
> > non-priv users even when permissions would permit. That option is also
> > non viable. Nice idea. Guess I know why none of us pursued it now.
> ... But that's why we bind mount. Sorry, I've still not dived deeper,
> into the code, but something here isn't making sense to me. The
> unprivileged user cannot create the /dev/.lxc/user-whatever directory to
> which he was trying to symlink, so what should create that?
Actually, he can. /dev/.lxc/user is set up mode 1777 much like /tmp
specifically for that purpose. So an unpriv user gets a
directory /dev/.lxc/user/{container-hash} owned by him and restricted to
him while a priv user gets /dev/.lxc/{container-hash}. That's done in
the startup helper script that set up the parent /dev/.lxc directory.
> I assume
> that in fact you want the persistent container-/dev for an unpriv user
> to actually come from ~/.local/share/lxc/container/root.dev ? Then,
> yes, inside there when starting the container, for any file which exists
> under ~/.local/share/lxc/container/root.dev lxc should bind-mount from
> the host's /dev.
I don't think the autodev populating code currently does bind mounts.
If not, it probably should. Probably should look into that. :-P
> -serge
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20141001/6ab252e2/attachment.sig>
More information about the lxc-users
mailing list