[Lxc-users] container shutdown

[Brian K. White]

On 3/19/2012 9:25 AM, Serge Hallyn wrote:
> Quoting Daniel Lezcano (daniel.lezcano at free.fr):
>> On 03/19/2012 03:50 AM, Serge Hallyn wrote:
>>> Quoting Daniel Lezcano (daniel.lezcano at free.fr):
>>>> On 03/19/2012 12:00 AM, Serge Hallyn wrote:
>>>>> Hi,
>>>>>
>>>>> Thanks to Jäkel's and Fajar's great ideas, we can now cleanly shut down
>>>>> a container by sending it SIGPWR.  I'm attaching two ways to do that.
>>>>> In-line is a patch which modifies lxc-stop to take optional -s and -t
>>>>> args - -s for shutdown (meaning send SIGPWR), and -t for a timeout,
>>>>> after sending SIGPWR, to hard-kill the container.
>>>> That may make more sense to implement a lxc-reboot | lxc-shutdow
>>> Is there another signal that would make sense for lxc-reboot?
>>
>> Yes, SIGINT will make the init process to restart the services. I
>> said lxc-reboot but that could be lxc-shutdown -r.
>
> I personally prefer lxc-reboot, but I can imagine people liking
> lxc-shutdown -r.  What do others prefer?
>
>>>> script on top of on lxc-kill.
>>>>
>>>> IMHO, I don't think adding a timeout is a good idea because the
>>>> shutdown process may take more than the timeout to stop the services
>>>> and the container could be killed while the services are doing some
>>>> cleanup or flush or whatever. If this option is present, people will
>>>> tend to use it instead of investigating if a service is stuck, or
>>>> working, or flushing.
>>>> I would recommend to let the shutdown script to handle the timeout
>>>> by themselves.
>>> By 'let the shutdown script to handle the timeout by themselves", you
>>> mean let the scripts calling lxc-shutdown handle the timeout?
>>
>> I meant the initrd scripts within the container to be fixed to
>> properly shutdown (for example add timeout or optimize the stopping
>> services). The init process will send SIGTERM to all the processes
>> and then SIGKILL after awhile. I don't think that should be handled
>> from outside.
>
> I agree we want to do that where we can.  I disagree that we should
> rely on it.
>
>> Some services are bogus because they don't care when
>> they are stopped in the shutdown process because they expect to be
>> killed. For example, the sshd service was automatically respawned
>> after being killed by init at the shutdown time but that was only
>> spotted with containers.
>
> Right, and we should (and did) fix that, but lxc shouldn't look
> broken when the container misbehaves.
>
>>> leave lxc-shutdown to be as simple as 'lxc-kill -n $1 SIGPWR" ?
>>
>> Yes, lxc-shutdown could be in this case very trivial (may be adding
>> a couple of things like waiting for the container to stop before
>> exiting in order to have a synchronous command).
>
> (I dunno, from there it seems to me the next logical step to add a
> timeout :)  But just waiting is fine for me.)
>
> Ok, so
>
> lxc-kill -n $1 SIGPWR
> lxc-wait -n $1 STOPPED
>
> I'll wait for comments on lxc-reboot v lxc-shutdown -r.

Timout:
I can think of no excuse to omit a timeout option. It would be easy and 
it would be useful and it would be more admin-friendly than requiring 
the init script author to do it, or fail to do it, or do it poorly, or 
have 12 different distro's all do it differently, etc...

Any that want to do it themselves, still can, since it's merely an 
option not a hard coded behavior. If you need to watch for something 
that _you_ know means it's ok to destroy, yet doesn't look like 
"stopped" to lxc-wait, no problem, just don't use that option.

But by far the more usual and therefor should be the default behavior, 
would be don't allow a hung container to prevent the host from shutting 
down gracefully. That allows one bad container to possibly harm the host 
and thereby all other containers on that host. My own init script for 
suse has this problem. I know ways I could fix it but I've just been 
busy with other work so it just continues to have this problem for a 
year now...

Executable name:
I would prefer several almost identical actions to be implemented in one 
program with options instead of several almost identical programs. So I 
say lxc-shutdown -r than lxc-reboot. But I have no problem with 
lxc-shutdown doing -r based on argv0 as well as getopts. Everyone can 
have what they want without asking you the author to write multiple 
programs.

-- 
bkw