[lxc-devel] lxc-create and template API issues

[Valentin LAB]

Hi,

I'm having troubles migrating our lxc template from ubuntu 12.04 basic 
install to new 14.04 version. One of the major differences is that we 
can't lxc-attach or lxc-start the LXC being built. I need to do this to 
launch several startup scripts in the LXC to finish the set up.

This request seems quite normal to me, and other templates seems to 
struggle to do the same. Mostly using chroot.

For example, I see that lxc-fedora does some chroot... but this requires 
to make some mount (for /dev and /proc). And if anything goes wrong 
(Ctrl-C from the user, or the script does not finishing well for any 
reason (server unreachable), lxc-create will try to "rm -rf" the 
container rootfs (because the template script returned a non 0 
status)... leading to remove all in /dev or trying to remove things in 
/proc... And this will remove /dev/* on the host also. Yikes !

Ensuring having the template return 0 status in case of error just to 
avoid the big nasty deletion is perverse at best, or to umount properly 
these devices is cumbersome and error prone (and this even if all went 
right in the script you've launched on the LXC): Any "apt-get 
install..." could spawn services that get re-parented to PID 1 and 
they'll hook up to files in "/dev" and "/proc"... This means that umount 
will be refused unless you manage to pin and kill every process that 
were created in the chroot, and this is not trivial.

This means that the lxc-fedora could have a big nasty bug leading to 
deleting content of /dev/* on the host if you happen to Ctrl-C at the 
bad moment... (I didn't check this). Of course you can rebuild /dev on 
most system quite easily, but if you don't do it, your host system is 
broken by simply launching lxc-create -t fedora and cancelling it at the 
bad moment...


The best solution I have found so far is to move the new "partial" file 
location in /var/lib/lxc/my_container/partial, so that lxc-start work 
again, and let me start the container and lxc-attach some commands.

Of course, all this feels really wrong.

What am I doing wrong ? Do you have any suggestion ?

I must add that I use some sort of configurable templates that allows me 
to feed shell scripts (pretty like juju charms hooks/install) to build 
different services at lxc-create time. This worked very well in 12.04. 
The scripts are various and are meant to be run inside the container to 
set-up specific services automatically.

Thanks for your insights,

-- 
Valentin LAB