[Lxc-users] lxc-start-ephemeral + upstart = pain?

[Serge Hallyn]

Quoting Dan Kegel (dank at kegel.com):
> Hi ho.
> ( I see https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1049908 also
> deals with lxc and upstart, but my use case is probably different.)
> 
> I'd like to run an ephemeral container in a respawning upstart job,
> but I don't know how shutdown should work.
> 
> Has anybody else tried this?
> 
> First issue: upstart likes to use signals for this, not commands.
> Looking at lxc-restart and lxc-start, I noticed that lxc already does
> use signals for shutting down.  Whew.
> 
> Second issue: it's init that needs to get the signal, not lxc-start.  Happily,
> lxc-start seems to forward signals to init.
> 
> Third issue:
> upstart sends SIGTERM and then later SIGKILL, whereas
> lxc-restart sends SIGINT and then later SIGPWR.
> Bit of an impedence mismatch :-)
> Happily, Upstart lets you tell it which signal to send instead of SIGTERM,
> and even has
>   kill signal INT
> as an example .conf line to change that to SIGINT.
> (I don't know if it lets you configure the later signal.)
> 
> Fourth issue: which PID to kill?
> Upstart figures out which pid to send the kill to as follows:
>   default:  command is running in foreground, just kill it
>   "kill fork": command forks once, watch the fork and kill the child
>   "kill daemon": command forks twice, watch the forks and kill the grandchild
> My script which runs lxc-start-ephemeral, which runs lxc-start and then ssh:
>   ””€sh /home/buildbot/buildbot-git/slave/lxcslave.sh
> start_ephemeral_once_foreground hello
>   --> ”€bash -x
> /home/buildbot/buildbot-git/slave/cherrypicks/lxc-start-ephemeral
> --overlaydir /ephemeral-overlay --name ...
>   -----> lxc-start
>   ----------> init
>   -----> ”€ssh -n -o StrictHostKeyChecking=no -o
> UserKnownHostsFile=/dev/null -i /home/buildbot/.ssh/id_rsa
> buildbot at 10.0.3.224 --...
> So perhaps if I tell upstart to watch for the second fork (with
> "expect daemon"),
> it will choose lxc-start as the process to kill.
> But no.  It turns out that upstart counts all forks, including any
> commands before the final daemon is started, so you *can not* use a
> shell script that runs any external commands before starting the final
> daemon!  ( See https://bugs.launchpad.net/upstart/+bug/530779/comments/10
> )
> Since lxc-start-ephemeral does all kinds of forks before lxc-start,
> there's no way upstart can find lxc-start by counting forks.
> 
> Might be able to work around this by wrapping lxc-start-ephemeral in a
> program which forks off the script, then sits waiting for signals
> (SIGCHILD or SIGINT), and tell upstart to send its signals to the
> wrapper.
> 
> Oy, gevalt!
> 
> I wonder if the Python version of lxc-start-ephemeral might help here.
>  Or is sh able to do what I need with the proper use of & and friends?

Just thinking out loud - what about creating a wrapper script which
traps SIGUSR1, having it start your ephemeral container and then lxc-wait
on it to be first RUNNING then STOPPED, have the upstart instance job
leave command running int he foreground, have it 'kill signal SIGUSR1',
and then have the script lxc-stop when it gets SIGUSR1?

-serge