[lxc-devel] [RFC] Unified message bus for containers

[Andrian Nord]

Problem:
Currently there is at least two implementations of message passing
between hostspace and containerspace - af_unix and abstract socket.
First is used for commands and second - for state-reporting.

This is too much, imo. Also, second type is not thread safe.

Suggestion:
As container started, we always have at least two processes running -
first is lxc-start in hostspace and secord - forked process into
containerspace. As lxc-start is always exists when container is running,
it should be used as communication gate for hostspace applications, i.e.
it should export af_unix socket into some dir (/var/run/lxc, for example),
and listen it for commands from outer world. State could be also
reported in most of cases by lxc-start. Is lxc-start was shutted down
properly - it should remove it's socket, so no socket == container
stopped. If there was some fail and socket still exists, but no other
side connected (no lxc-start process) == container crashed. Also, this
will implement name-locking as well, as sockets, obviously, should be
named by container's name. If such socket already exists, lxc-start
should try bind to it (if old container had crashed), if success - try
request it's state - it could be stopping already, so lxc-start should
wait until it will stop (lxc-start should notify all listeners, that
it's stopped before quiting, so they may unbind) and free socket.
lxc-stop should behave in similar manner.

I'm suggesting to use pipe for communication between lxc-start and forked
process. Before execve forked process should always fork(clone?) some
supervisor process and proxy it's (piped) stdin to supervisor's stdin,
and then execve - this way communication will not be lost after execing
requested application. Supervisor may be used for more precise state
reporting, mounting new fs (if it would be forked before chrooting),
running arbitrary commands.

P.S. As you may clearly see - first part, i.e. af_unix sockets and other
hostspace stuff should not be handled into liblxc, as liblxc may be
called by some kind of daemon, that tracks many containers at once and
don't need any external communication methods. liblxc should only supply
fd to communication pipe and functions which will work with it.

P.P.S. Actually, supervisor may be a function supplied to liblxc by upper
packages - this will simplify library code, as library will not mess
with actual commands, protocol and command handling into supervisor.
Also, if no command-to-run was specified, library may just run supplied
function (so supervisor will be pid 1), which would allow upper package
provide it's own init realisation.