[lxc-devel] lxc-create and template API issues

[Serge Hallyn]

In my package builder, what I do is start the container, (read
a fifo in the container to which an upstart job writes when runlevel
2 is reached), attach and complete the configuration I want.

We currently have clone hooks.  We could add create hooks, which
could be specified by both the template and the user;  however
those currently are executed in a new mount namespace (so mounts
get cleaned up) but not with the container's networking.

I think it'd be good to implement create hooks which are executed
inside a firstboot of the container.  Now of course you can do this
(as I did in previous versions of my package builder) by having an
upstart/init job in the container rootfs, so that it simply gets
executed on the first boot.  But we could have lxc do this for you.

We would specify that the template itself must initialize the
container to the point where it can start up and set up a basic
network.  After running the template, lxc would then start the
container, wait for it to be ready, attach, and run the create
hook.

-serge

Quoting Valentin LAB (valentin.lab at kalysto.org):
> Hi,
> 
> I'm having troubles migrating our lxc template from ubuntu 12.04
> basic install to new 14.04 version. One of the major differences is
> that we can't lxc-attach or lxc-start the LXC being built. I need to
> do this to launch several startup scripts in the LXC to finish the
> set up.
> 
> This request seems quite normal to me, and other templates seems to
> struggle to do the same. Mostly using chroot.
> 
> For example, I see that lxc-fedora does some chroot... but this
> requires to make some mount (for /dev and /proc). And if anything
> goes wrong (Ctrl-C from the user, or the script does not finishing
> well for any reason (server unreachable), lxc-create will try to "rm
> -rf" the container rootfs (because the template script returned a
> non 0 status)... leading to remove all in /dev or trying to remove
> things in /proc... And this will remove /dev/* on the host also.
> Yikes !
> 
> Ensuring having the template return 0 status in case of error just
> to avoid the big nasty deletion is perverse at best, or to umount
> properly these devices is cumbersome and error prone (and this even
> if all went right in the script you've launched on the LXC): Any
> "apt-get install..." could spawn services that get re-parented to
> PID 1 and they'll hook up to files in "/dev" and "/proc"... This
> means that umount will be refused unless you manage to pin and kill
> every process that were created in the chroot, and this is not
> trivial.
> 
> This means that the lxc-fedora could have a big nasty bug leading to
> deleting content of /dev/* on the host if you happen to Ctrl-C at
> the bad moment... (I didn't check this). Of course you can rebuild
> /dev on most system quite easily, but if you don't do it, your host
> system is broken by simply launching lxc-create -t fedora and
> cancelling it at the bad moment...
> 
> 
> The best solution I have found so far is to move the new "partial"
> file location in /var/lib/lxc/my_container/partial, so that
> lxc-start work again, and let me start the container and lxc-attach
> some commands.
> 
> Of course, all this feels really wrong.
> 
> What am I doing wrong ? Do you have any suggestion ?
> 
> I must add that I use some sort of configurable templates that
> allows me to feed shell scripts (pretty like juju charms
> hooks/install) to build different services at lxc-create time. This
> worked very well in 12.04. The scripts are various and are meant to
> be run inside the container to set-up specific services
> automatically.
> 
> Thanks for your insights,
> 
> -- 
> Valentin LAB
> 
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel