[Lxc-users] [lxc-devel] Working LXC templates?

[Michael H. Warfield]

On Mon, 2013-09-09 at 08:58 +0200, Natanael Copa wrote: 
> On Sun, 08 Sep 2013 20:33:16 -0400
> "Michael H. Warfield" <mhw at WittsEnd.com> wrote:
> 
> > With all due respect...
> > 
> > On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote: 
> > > After putting some thought into this,
> > > IMO LXC badly needs a universal tool with the following features
> > > 
> > > - A single script should be used to run on <any> HostOS, creating
> > > <any> supported Container OS. Although this would make the script
> > > extremely large, IMO it would actually be easier to maintain in the
> > > long run.
> > 
> > Actually, no.  From my experience (30+ years in software development),
> > it would turn into a morass.
> > 
> > The problem here is that the maintainer(s) would then need to understand
> > how each and every distribution is installed and how it would be
> > installed on each and every distribution.  It would distill the worse of
> > all the problems we have now in the templates into one great big dung
> > pile.  It would rapidly become unmaintainable.  The "extremely large" is
> > the red letter warning that it will become unmaintainable as fewer and
> > fewer people understand what this great big huge blob does.

> I tend to agree with this.

> What I do think could be done is to use template APIs. Either by having
> a script "library" with helper functions (for things like:
> generate_mac_address etc) or let the template scripts be "plugins" that
> must provide a set of pre-defined functions (eg. install_distro,
> configure_distro, copy_configuration etc) or maybe a combination of
> those two.

I like this idea, it's just that, in the short term, we have to get past
this one gotcha.

In the git-stage current Fedora template, the entire problem is embodied
in the "download_fedora" function starting around line 201...  The
gotcha's are three commands around line 272 after we've identified and
downloaded the initial release rpm.  We have this:

    mkdir -p $INSTALL_ROOT/var/lib/rpm
    rpm --root $INSTALL_ROOT  --initdb
    rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
    $YUM install $PKG_LIST

Ok...  Those are running in the host local run time environment.
Obviously, if the host does not have a (compatible) version of rpm
and/or yum in the host local environment, you're screwed.  That's the
catch-22 situation and it's the same situation with zypper in the
OpenSuse template.  That short space of code has to be recreated in a
totally distro-agnostic manner so it runs on any distro to create our
initial rootfs.  After that, we can do what ever distro (Fedora)
specific commands we want by chrooting into the target container first
(including rebuilding the rpm database to the target version).  That's
only even needed if you don't already have a cached rootfs for that
distro and version.

I was SOOOO close last week working on this while on vacation.  Recent
revs of Fedora have this downloadable LiveOS core that runs on the
netinst CD and others.  That's the 200MB - 300MB blob I was referring
to.  You just download it, mount it (requires squashfs) to a directory
and then mount the ext3 image it contains on another directory.  Then
create and bind mount a few rw directories (etc var run), mount proc in
the image, and bind mount your rootfs to run/install in the image.  Then
chroot into the image and voila, instant RTE.  Yum and rpm are capable
of installing other versions of Fedora, so I wouldn't (shouldn't) even
need a version specific instantiation of the RTE, just one that can
install every version we might be interested in.

Except...  It didn't work.  Sigh...  $#@$#@#@

Fedora came so close and then face planted for what I wanted to do.
Sure rpm is in there.  But rpmdb is not.  So, no --initdb and no
--rebuilddb.  They've got yum in there, but no yummain.py so yum hurls
chunks immediately upon execution trying to import yummain.  What the
hell good does it do to have yum in there but no yummain?!?!  But,
something, fixed up, like that from all the distros would be a perfect
answer (albeit somewhat less than high performance thanks to that
download, but it's a single download that could be cached).  The only
potential gotcha I see in there is requiring squashfs available for
mounting the image.  Anyone have heartburn over that?

That squashfs image has to be able to do an install or it would be
useless on the netinst CD.  I'm not sure if they still have anaconda on
there or not but it has to be able to do a kickstart install, so all I
need is a minimal.ks kickstart file to perform essentially an unattended
install of a minimal build into the target rootfs.  That's where I'm at
now, trying to get that to work.

But it's all that work just to replace those few lines of code above
where you can not perform a host native install of the rootfs.  That's
all that's standing in the way and it's frustratingly close for the
Fedora template.

We've got the following distro templates in the project, currently:

lxc-alpine
lxc-alt
lxc-arch
lxc-busybox (is this really a distro template - I'm not sure)
lxc-debian
lxc-fedora
lxc-opensuse
lxc-oracle (really?)
lxc-ubuntu / lxc-ubuntu-cloud

Who are the owners?  I don't "own" the Fedora template but I work on it.
Each template owner has to solve this problem in their own way and it
all boils down to the same problem.  Building that initial rootfs in the
host native environment.

After that, the idea of generalizing the functions into an API form
sounds like a great idea.  It could be that this catch-22 could be
buried in a "create_rootfs" function which would create a cacheable
rootfs.

I would love to see templates for CentOS, Scientific (SL), Slackware and
maybe even RedHat EL.  NST (Network Security Toolkit) and Kali (fka
BackTrack) would also be nice and I might work on them (which can I say,
I'm a security fanatic).  But I gotta solve this problem first in a way
I can use for the ones I might want to write down the road.

But, in the meant time, we have "the recipe for rabbit stew"...  First
you catch your rabbit.  First we have to have a way to build the initial
rootfs.

<aside>

This entire process reminds me of when we (I) had to bootstrap gcc into
environments (HPUX and AIX years ago) which had no C compiler installed.
First you cross compiled a compiler (on Solaris in my case) for your
target arch and transported it there.  Then you used that "stage 1"
compiler to compile the gcc compiler on the native platform (stage 2).
Then you used the stage 2 compiler to compile a full stage 3 compiler
where you could then confirm the outputs from the stage 2 and stage 3
compilers to confirm they matched (which never seems to happen the first
time you try it).  When you were done and finally confirmed working, you
really needed a beer.  This is the same sort of bootstrap process.

</aside>

> -nc

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20130909/a1b60d22/attachment.pgp>