[Lxc-users] Dynamic devices...
Michael H. Warfield
mhw at WittsEnd.com
Sun Mar 7 17:50:33 UTC 2010
On Sat, 2010-03-06 at 23:21 -0600, C Anthony Risinger wrote:
> What does udev listen to while under container (or host)? Is it
> possible to generate events from a host process to each containerized
> udevd, and let udevd do it's job? I'm not familiar; a "spoofed"
> kernel event generator of sorts
You know, I was thinking about that, exactly. I'm not totally sure, but
it just seems like that would almost work. Then you COULD have udev
rules in the container that did things that were different from the host
and you would not be constantly fighting with driving a stake through
udevd's heart in the containers (the containers hang stone cold dead in
system initialization).
Daniel has it right, though... This is part of the network namespace
and needs to be really handled at the kernel level. And this is a
problem shared by both LXC and OpenVZ (and probably Vservers and most
certainly libvirt and their "lxc" driver). Then we really really could
virtualize even udev effectively. I'm not totally sure how effective
spoofing the udev events from the host to the container would be. I
would say, tricky, to say the least. You would have to keep from
screwing up any other netlink kinds of things that are overloaded on
that interface. You'd have to be operating on the udevd level in the
host and understand how to separate out those events for the various
containers and then ship them off to the emulated netlink interfaces.
Doable? Probably. For some value of doable.
On top of that, as Daniel pointed out, you have the issue of the static
major/minor device acls in the LXC configs. I'm not sure how you would
deal with that, but it needs to be dealt with with or without the
hotplug udev issues. In my case, the usb sharing device has a fixed
major and minor because none of the servers have any other hid devices.
But, what if I plugged in a USB keyboard or mouse while I was on-site
(which does happen once in a lllooonnnggg while) and THEN switched the
console? Ooopppsss... No workie. Minors are now different. So I have
to allocate ALL of that major. Same thing with those USB tty's only
worse and I have to deal with that right now. Two of the systems have
static serial adapters going to X10 controllers that control power to
the various servers. The other two do not. Guess what... The minor
numbers are now different between those systems. I can't assign just
ttyUSB5 to "Onyx" and ttyUSB6 to "Opal" and expect to be able to migrate
those containers between the various host engines that have with and
without. I have to allow the entire major and trust those admins
(which, fortunately, I can) to NOT go screwing with each others consoles
(which they would need the root passwords to the servers, which they
don't anyways). But there are certainly worse scenarios which are easy
to envision where you would want tighter restrictions.
I now have a udev hotplug script, /lib/udev/lxc-hotplug, that seems to
be really close (I know, I owe a lot of people some posting of some of
my big hairball scripts already) to what I want and is working for me
now. I'm storing a lot more stuff under /var/lib/lxc/${VM} (like an
entire dynamic udev dev tree) to make it work without committing heinous
random acts of terrorism (like, if you don't create it DON'T REMOVE
IT!!!), but it seems to work pretty nicely and it mirrors the host
behavior into configured selected containers. But it still DOESN'T
allow for custom udev events and scripting (which is fine by me) in the
containers. Maybe, once I clean up a lot of the debugging cruft and
minimizing the "mikey"ism's, I can post what I've done. May be of some
use to others or as yet another fine example of what not to do and what
to avoid. :-)
Mike
> On Mar 6, 2010, at 11:49 AM, "Michael H. Warfield" <mhw at WittsEnd.com>
> wrote:
>
> > Hey all,
> >
> > This is sort of a jump ball for both the OpenVZ camp and the LXC camp.
> > Maybe some food for thought as well. I think it's something that
> > needs
> > to be thought out.
> >
> > I know udev does not work in containers. Neither OpenVZ nor LXC. Ok,
> > fine. So devices are static. Problem is that I've got a situation
> > where I need dynamic devices, specifically some USB devices.
> >
> > I have an 8 port serial to USB converter used for controlling the
> > serial
> > consoles of a cluster of remote servers. So ttyS0 on each server is
> > hooked to a port on the controller and configured for serial console
> > (they also have serial BIOS console redirection as well - very VERY
> > nice
> > for remote management where you don't have a remote IP KVM).
> >
> > That converter then plugs into a USB sharing device which shares the
> > converter between 4 of the servers. So... Any one of the four
> > servers
> > can take control of the converter and can then access all of the 8
> > serial consoles for all of the servers. Nice redundancy.
> >
> > I want to grant access to that subsystem to selected containers so
> > certain administrators can be given their very own container and be
> > able
> > to access the host consoles over serial ports, then. Simple so far.
> > Just allow those containers access to the usbsharing device and the
> > ttyUSB* devices.
> >
> > Here's where the rub comes in. The way the USB sharing device works.
> > Any server which does NOT have the consoles sees a USB hiddev device
> > (it
> > looks like an USB human input device). It accesses that device to
> > request (or demand) control of the controlled USB devices. The server
> > with the consoles, does NOT see that device at all. What it does see
> > are the controlled USB devices (the ttyUSB* devices in this case)
> > which
> > the other servers do NOT see. So the logic goes that if you see the
> > usbsharing device, you do not have the consoles but may request
> > them. If
> > you do NOT have the usb sharing device you should have the ttyUSB*
> > devices and may access them. So those devices come and go
> > dynamically.
> >
> > How to do this in a container?
> >
> > (Yes, obviously, I can try to open the device and get an error. But,
> > really... That's an ugly answer.)
> >
> > I suppose I can apply the dynamics in the udev rules in the host
> > machine
> > and create matching devices in the container's dev directory using a
> > hot
> > plug run script. That's what I'm working on now. Is that really the
> > answer though?
> >
> > Just food for thought for developers and users alike. There has to be
> > more situations that just this.
> >
> > Regards,
> > Mike
> > --
> > Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> > NIC whois: MHW9 | An optimist believes we live in the
> > best of all
> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure
> > of it!
> > ---
> > ---
> > ---
> > ---------------------------------------------------------------------
> > Download Intel® Parallel Studio Eval
> > Try the new software tools for yourself. Speed compiling, find bugs
> > proactively, and fine-tune applications for parallel performance.
> > See why Intel Parallel Studio got high marks during beta.
> > http://p.sf.net/sfu/intel-sw-dev
> > _______________________________________________
> > Lxc-users mailing list
> > Lxc-users at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/lxc-users
>
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20100307/3e1f83df/attachment.pgp>
More information about the lxc-users
mailing list