[lxc-devel] Dynamic devices

[Michael J Coss]

I'm using Gentoo as my host OS (3.8.0 kernel), and the latest ebuild of 
lxc is 0.8.0.  Now it's possible (probable) that the udev that I'm 
running is different than what is running on Ubuntu, but the behavior on 
my system is that no devices are created.   And yes, I've opened the 
capabilities to allow the container to create devices.  The uevent 
messages are received, as I noted before but by not acted upon.

But getting back to the question of policy, does it make sense that the 
way policy is implemented is that the all containers receive the events, 
and container configuration determines what uevents should/can be 
processed by that container.  Or  should it be handled elsewhere?

----Michael J Coss

On 3/12/2013 1:04 PM, lxc-devel-request at lists.sourceforge.net wrote:
> Send Lxc-devel mailing list submissions to
> 	lxc-devel at lists.sourceforge.net
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://lists.sourceforge.net/lists/listinfo/lxc-devel
> or, via email, send a message with subject or body 'help' to
> 	lxc-devel-request at lists.sourceforge.net
>
> You can reach the person managing the list at
> 	lxc-devel-owner at lists.sourceforge.net
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Lxc-devel digest..."
>
>
> Today's Topics:
>
>     1. Re: Dynamic devices (St?phane Graber)
>     2. [PATCH/RFC] Add lxc.stopsignal config option
>        (Alexander Vladimirov)
>     3. Re: [PATCH/RFC] Add lxc.stopsignal config option (Serge Hallyn)
>     4. [PATCH] oracle template: fixes for older releases (Dwight Engen)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 11 Mar 2013 17:19:27 -0400
> From: St?phane Graber <stgraber at ubuntu.com>
> Subject: Re: [lxc-devel] Dynamic devices
> To: lxc-devel at lists.sourceforge.net
> Message-ID: <513E4A5F.3010408 at ubuntu.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On 03/11/2013 04:49 PM, Michael J Coss wrote:
>> I know that this has probably been hashed over dozens of time but as far
>> as I can tell udev still does not work properly in containers, neither
>> OpenVZ nor LXC variants.  Unfortunately, I really do have a need for
>> dynamic devices, specifically some USB devices tunnelled over IP.  I'm
>> looking at using LXC as the basis for a cloud computing infrastructure,
>> where user & device migration needs to allow for a user to start up a
>> container and "bind" an arbitrary set of devices to that container (as
>> hotplug events).  I like LXC for it's low overhead, and apparent
>> upstream support, and had hoped that with the introduction of the
>> various namespaces (user, network, etc.) that a udev solution would have
>> arisen.
>>
>> Of course, most google searches on this topic, say "udev doesn't work",
>> "don't install it", or "remove it from init.d", etc.
>>
>> On the one hand, there is the case to be made that containers should
>> simply be pre-provisioned with all devices that are needed by that
>> container.  However, I can imagine a operating model, where the host
>> system applies a policy on the kobject events, and forwards them to the
>> appropriate container.  Alternatively, the host could apply the policy,
>> but instead of forwarding the events to the container, it could simply
>> process the events from the "host" side, modifying the dev directory for
>> the appropriate container.  Or events could be sent to all containers,
>> and the policy as specified in the lxc configuration would limit what
>> devices can or can not be created. Or finally, the policy could be
>> applied in the kernel where the event broadcast occurs.
>>
>> I discount the last as this kind of security policy really doesn't, in
>> my opinion, belong in the kernel. But at the end of the day, I really
>> would like not be constantly fighting dependencies on having
>> udev/systemd installed in the containers, as well as supporting dynamic
>> devices.
>>
>> When I first looked at the kernel, it looked to me like there is some
>> attempt at isolation of the uevents.  There is a bcast filter function
>> which looks like it would only forward events to listening processes
>> (udevd) if they were in the same network namespace.  But experimentally
>> this does not seem to be the case.  On my setup, running a Gentoo host,
>> and a Gentoo container, if I run udevadm monitor on both host and
>> container, I see the kobject uevents on both systems.  What happens
>> after that is a bit of a mystery, as I haven't delved into why the
>> container udev doesn't seem to properly process the event.
>>
>> Each approach seems to have some merit.  If you forwarded the events to
>> the containers udevd via netlink socket, then the container's udevd
>> could in theory be the stock udev, no modifications necessary.  Only the
>> host's udev would need to be modified.
>>
>> Going down the other path, you'd need to ensure that no kernel uevents
>> are ever forwarded to a container namespace.  This can be done via a bit
>> of a hack although I'm still curious why events leak to the containers
>> when they are in separate namespaces.
>>
>> Before I go too much further down this road I was wondering what the
>> current consensus is and figured this was the place to ask.  So
>> thoughts?  comments?
>>
>> ---Michael J Coss
> The short and rather usual reply to this is that we know we'll need a
> device namespace at some point. Exactly how it'll work is yet unknown
> and it's never been high enough priority that anybody really worked on it.
>
> However I'm a bit surprised by your e-mail because in Ubuntu we
> certainly run udev in all containers and I see the devices under /dev
> (and /dev/bus/usb in your case) get created as is expected.
>
> Access control will obviously prevent me from accessing them unless they
> are allowed in the container's config though.
>
>
> So I'm not really sure what version of LXC and on what distro you're
> doing your tests, but on recent Ubuntu, we definitely do run udev by
> default in our containers and device nodes get dynamically created as
> uevents are forwarded by the kernel.
>
>