[lxc-devel] Container autostart proposal

Tue May 28 15:47:40 UTC 2013

Hey all...

On Tue, 2013-05-28 at 09:19 -0400, Stéphane Graber wrote: 
> On 05/28/2013 03:14 AM, Natanael Copa wrote:
> > On Mon, 27 May 2013 15:07:03 -0400
> > Stéphane Graber <stgraber at ubuntu.com> wrote:
> > 
> >> Hello,
> >>
> >> One feature that distros have been hacking together on their side for a
> >> while is container autostart.
> > ...
> > 
> >> So I therefore have two proposals on how to implement this, let me know
> >> what you prefer:
> >> 1)
> >>  - Keep things as they are in Ubuntu and make that the official way of
> >> auto-starting containers. Documentating /etc/lxc/auto as the location in
> >> which people need to place their symlinks.
> >>  - Update our tools to properly list and destroy those auto-started
> >> containers.
> >>  - Extend lxc-start and lxc-stop to allow starting all auto containers
> >> and stopping all known containers (-a flag would be my suggestion).
> > 
> > The nice thing with this is that it is simple and stupid.
> >  
> >> 2)
> >>  - Quite a bit more complex but more flexible.
> >>  - Introduce a new lxc.autostart system config option (lxc.conf).
> >>    This is a boolean turning autostart on/off on a system level.
> >>  - Introduce a few more container options:
> >>    + lxc.start.auto => boolean
> > ...
> >> start the first one, wait up to lxc.start.time for it to be RUNNING,
> >> then start the next one, etc...
> >>  - The defaults would be lxc.start.auto=false, lxc.start.priority=0,
> >> lxc.start.time=0.
> >>  - It'd be assumed that only containers in lxc.lxcpath can be autostarted.
> >>  - This design would also work for unprivileged containers where
> >> lxc.lxcpath points to the user's home directory, in such case, the
> >> distros will need to have something call "lxc-start -a" on session open
> >> and "lxc-stop -a" on session close (or whenever they want).

> >> While 1) is clearly a lot less pain and much easier for the distros
> >> already supporting autostart to migrate to, I think there's a pretty big
> >> benefit in getting 2) as it's more flexible, allows for delayed start
> >> and supports unprivileged containers.
> >>
> >> Thoughts?

First off.  Set your wayback machine Mr Peabody.  This did come up a
couple of years ago (plus or minus a year, around post 0.7.5 time
frame).  At that time, I became aware of the Ubuntu convention of the
autostart directory and links.

There was some discussion over boot priority and Serge mentioned
something about some "prefix" to the container names which could be used
in those links to provide "priority" or boot order for order dependent
systems in the Ubuntu system.

At that time I voiced my opinion that I personally liked the OpenVZ
paradigm of having an "autoboot" and "disabled" flag in the conf files.
Containers while "autoboot" set and "disabled" false where booted when
the sytem booted.  Containers with "autoboot" false and "disabled" false
could be started with a standard startup command (which included some
wildcarding).  Containers with "disabled" true could not be started
unless "forced" (this allowed for experimental, potentially destructive
or unstable, containers to exist and would not be started by accident).

I still like this system.  I also suggested an explicit priority in
there to control boot order.

Options 1 and 2 are NOT mutually exclusive.  They can work together
without conflict.  At the time, I envisioned an autoboot system that
could support both of those paradigms.  The existence of a
link /etc/lxc/autoboot merely implies (or defaults) lxc.autoboot to true
(which could then be overridden in the actual config file).  You
basically enumerate the containers to be autobooted based on certain
criterion.

What I thought of would flow like this...

1) First list out all containers in /etc/lxc/autoboot in lexigraphical
order (Ubuntu paradigm) and give them a certain default "priority" (lets
say 100 or some configurable value).  Values of "autoboot" in the
container are ignored (assumed true) or could override this (autoboot =
false) but any value for lxc.autoboot.priority would override the
default priority. 

2) Any container not captured in #1 in /var/lib/lxc with lxc.autoboot
"true" but no lxc.autoboot.priority is then added to the list with the
default priority while any container with lxc.autoboot "true" and with a
priority is added at the specified priority (which could be lower or
higher than the default).

3) Sort by priority and container (link) name and proceed to boot them.

The logic to enumerate the containers into a list based on the above
criterion is not that complicated.

A start delay time was also discussed which would introduce a time delay
after starting a container before moving on to the next.  This is going
to be critical with systemd containers due to the massive startup
parallelism that takes place (I've seen my startup load average in the
20s and 30s with a number of containers starting at once).

Priority also has the advantage of setting a boot threshold (only boot
containers > 99) but that could be done just as easily with groups and
those might be more versatile now that I think about it.

> > It would be nice to have the concept of groups. Like
> > 
> >  lxc.start.group = default
> > 
> > Then we have an 'lxc-start -a [group]' where you can start all
> > containers in group 'default' in one shot.

> I think I'd rather have that as "lxc.group" as it'd probably be used by
> lxc-ls and a few of the other tools to allow you to list only a group of
> containers.

Yeah, I concur with that too.  The idea of container groups is a good
one and extends to more than just booting.  I like that.

> But yeah, I can see the potential of grouping containers especially in
> development environments that require multiple containers for a given
> project.

I really like that idea of an lxc.group.  That idea was not raised the
last time but I really like the idea for both booting and, as you say,
for things like lxc-ls and others.  That should be part of this whole
proposal and doesn't contravene any of the above.  I think it should be
part of it.

My .02 euro.

> > Then I think it would be nice if the init system could take care of
> > dependency handling and start up delays. Dealing with states,
> > dependencies and parallel startup can be complicated (think circular
> > deps, deadlocks etc) and the init system already does all that.
> 
> I have no plan to implement actually dependencies between containers as
> this indeed quickly becomes a nightmare and is way more suitable for an
> init system to do. However the implementation of priority + time makes
> it easy for someone to write a tool generating and updating those two
> values based on dependencies.
> 
> FWIW I'm a big supported of designing your services properly so they
> cope with missing related service and just wait until everything is
> ready. But I realize not everyone has the time to make sure all their
> services behave in a sane way ;)
> 
> > What would be nice though, is some convenient way to fish out the
> > config options. Like, get me a list of all containers that are in group
> > 'foo'. Get me the value for config option 'lxc.start.wait' for
> > container 'bar'. (looks like there is a get_config_item for this)
> 
> We've got some ways to do that (though not with great filtering) through
> the API. From the shell, I think you'd need to iterate through the
> containers and then use something like lxc-config to extract the config
> keys.
> 
> That reminds me, lxc-config really should be updated to allow parsing of
> both a container config and the system config (instead of just the
> system config as it currently does).
> 
> 
> > -nc
> > 
> 
> 
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service 
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
> _______________________________________________
> Lxc-devel mailing list
> Lxc-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-devel

-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!