[lxc-users] creating device nodes in unprivileged containers?

[Fajar A. Nugraha]

On Wed, Jul 1, 2015 at 8:46 PM, Tomasz Chmielewski <mangoo at wpkg.org> wrote:
> On 2015-07-01 18:08, Fajar A. Nugraha wrote:
>>
>> On Wed, Jul 1, 2015 at 3:38 PM, Tomasz Chmielewski <mangoo at wpkg.org>
>> wrote:
>>>
>>> Really not possible? How do people run debootstrap, pbuilder? These tools
>>
>>
>> Not as root inside an unprivileged container
>>
>>> are often parts of build systems, am I really the first one to try to run
>>> them in LXC?
>>
>>
>> pbuilder with fakeroot should work
>
>
> Unfortunately it doesn't:
>
> tomasz.staff.com at build01:~$ fakeroot /bin/bash
> root at build01:~# pbuilder-dist trusty i386 create
> (...)
> I: running debootstrap
> /usr/sbin/debootstrap
> mknod: '/var/cache/pbuilder/build/6474/./test-dev-null': Operation not
> permitted
> E: Cannot install into target '/var/cache/pbuilder/build/6474/.' mounted
> with noexec or nodev
> E: debootstrap failed
> W: Aborting with an error
>


Apparently there are multiple problems. So short version, you really
can't use containers with pbuilder

Long version:
- fakeroot depends on some environment variables. pbuilder-dist clears
them when calling debootstrap. You can work around this to some degree
using wrapper script for debootstrap (or edit pbuilder to force
fakeroot), but there are still other problems
- debootstrap can work fine with fakeroot (it doesn't need fakechroot
on unprivileged containers), but that would only work if you build the
same arch as your container (e.g. using debootstrap --arch=i386 on
amd64 container will fail)
- pbuilder-dist will try to mount some directories (e.g /proc). This
will not work by default on containers under ubuntu as the default
apparmor profile does not allow mounting, even on privileged
containers
- even if you force "lxc.aa_profile=unconfined" and "lxc.cap.drop=" on
cotainer config (it allows pbuilder-dist to mount /proc and /run/shm
successfully), pbuilder-dist will fail when mounting /dev/pts.
Mounting it with "-o newinstance" works, and currently you need to
edit /usr/lib/pbuilder/pbuilder-modules to add that

"fakeroot pbuilder-dist trusty amd64 create" works on my unprivileged
trusty amd64 containers after those modifications, but I would only
give access to that container to people that I explicityly trust.

-- 
Fajar