[Lxc-users] [lxc-devel] Working LXC templates?

[Michael H. Warfield]

With all due respect...

On Sun, 2013-09-08 at 16:08 -0700, Tony Su wrote: 
> After putting some thought into this,
> IMO LXC badly needs a universal tool with the following features
> 
> - A single script should be used to run on <any> HostOS, creating
> <any> supported Container OS. Although this would make the script
> extremely large, IMO it would actually be easier to maintain in the
> long run.

Actually, no.  From my experience (30+ years in software development),
it would turn into a morass.

The problem here is that the maintainer(s) would then need to understand
how each and every distribution is installed and how it would be
installed on each and every distribution.  It would distill the worse of
all the problems we have now in the templates into one great big dung
pile.  It would rapidly become unmaintainable.  The "extremely large" is
the red letter warning that it will become unmaintainable as fewer and
fewer people understand what this great big huge blob does.

I've written perl scripts that count 3K+ lines long and have to have
over 50% of the lines be comments just so people can read it (ok - it's
perl, what can I say).  The template system with small, individually
maintained, distribution templates are vastly more maintainable.  I'm
pretty good with the Fedora stuff.  I'm not about to tell the
maintainers of the Ubuntu or OpenSuse stuff how to do their thing.  We
can formulate rules and test each other's templates, but we can not
write or maintain them.  But, we can be good at maintaining our own
templates.

> Unknown at this point is how long it would take to load, but
> once loaded I would expect the script to run very fast.

Extremely doubtful, in my experience, and the "run very fast" is not the
point and not the source of your problems.  From a systems analysis
basis, a function which is run rarely can run very inefficiently if it
does the right thing every time and is superior to something that runs
efficiently but missing way too many corner cases and is a complex
gemish-gamash to maintain because it's a great big hairball.  I don't
care about "run very fast".  I care about "does the right thing" in this
case.  Plus, if it's the Fedora create that's busted, you're limited in
the domain of script code you have to look at, even if you are
unfamiliar with the code over all.
> - Flexible enough to support any packaging system, and perhaps even
> non-package systems if someone should present a strong case or is
> willing to put in the work

It'll never happen.  It would take a huge design team, conversant in all
those packaging systems (I'm pretty damn good in yum and rpm, so-so in
pkg and apt-get, and absolutely suck at yast and zypper) and then it
would be "design by committee".  A committee is a life form with six or
more legs AND NO BRAIN.

> - Use the available HostOS distro packages whenever possible. Do not
> require packages from other distros or cross-platform packages. IMO
> this means less reliance on supporting non-mainline packages, keeps
> package and functionality support simple, understandable and
> responsibility clear.

Agreed.  But that is an argument for a modularized system with small,
distro specific, components melded with a small modularized system with
distro independent components.

BTW...  You think trying to install Fedora on OpenSuse is bad?  Try
installing OpenSuse on Fedora (or Ubuntu).  It's a disaster.  That
template won't even come off the dime (no offense to the author - we
just don't have the tools in our distros).  Here's what you get...

-- 
[root at forest mhw]# lxc-create -n OpenSuse -t opensuse

lxc-create: No config file specified, using the default config /etc/lxc/default.conf
/usr/share/lxc/templates/lxc-opensuse: line 369: type: zypper: not found
'zypper' command is missing
lxc-create: failed to execute template 'opensuse'
lxc-create: aborted
-- 

Epic fail.  There is no zypper on Fedora.

> - The script should be clearly modular, with sections clearly marked
> for the functionality in that section.

And that argues against one great big script (which would have to be
collectively maintained) and for smaller components (which could be
independently maintained).

> - The script as currently envisioned should be a single file to reduce
> missing "script dependencies." Any calls should generally be only to
> internal functions or to mainstream packages provided by the distro.
> Separate custom scripts should be avoided because otherwise the
> dependency script(s) may be hard to locate.

Totally disagree as my experience leads me to believe this ultimately
leads to bit-rot and deterioration in the maintainability where nobody
knows who wrote what line and why and nobody will touch it to avoid the
blame of breaking someone else's code.

Nope...  Not going to happen (IMNSHO).  I've run into this even with the
Fedora template.  Several times I've posted to the -devel list asking
who wrote this or that line and "what were you thinking" only to get no
answer.  Now raise that to the N'th power where N is the number of
authors (or lines) in a given file.

> To this end,
> I've done the intial baby steps towards architecting a new universal
> lxc-create template script that should have all the above features and
> likely within a week or so post an early alpha on my github for
> anyonne to criticize/contribute/modify.

Right now, I'm proceeding down an avenue that may lead me to generating
a template that should work everywhere.  The effort is to create an
intermediary RTE (Run Time Environment) based on the netinst (Network
Install) CD images present in all Fedora releases and many of the others
I have examined.  I may even resort to using the RedHat / Fedora
Anaconda kickstart system to create a minimal RTE rootfs and using that
to create the container cache rootfs.  The RTE could even be cached
alone with the individual roots for the containers.  It's decidedly NOT
efficient (about a 200MB-300MB download if you can not install
"natively") but it should work (still working out the bugs and
dependencies) but only has to be done once for all the containers of a
given distro.

Is this applicable to other templates?  Maybe yes, maybe no.  Can others
copy the technique of creating an intermediary RTE to facilitate the
initial build of the rootfs?  Hell yes (I hope so).  But, it's up to the
individual template maintainers.  At this point, I'm not sure if Serge,
Daniel, and Stephane and others consider me a maintainer for the Fedora
template or just an (very) annoying contributor (and agent provocatour)
but I wouldn't be offended or annoyed either way.

If you think you can do this, I'm more than willing to test and
critique.  Maybe you're better at this than I am.  Give it your best
shot.  If you can come up with a way to build the Fedora rootfs without
recourse to rpm or yum or anaconda or the netinst CD (or prebuild,
non-distro distributed tarballs), I would love to hear your thoughts.
And, no, just because you have rpm on OpenSuse, it doesn't mean it will
install Fedora, unfortunately.  One of the things I'm concerned about is
versioning in the rpm databases (on all rpm based distros) and if even a
distro RTE can build a true rootfs with the correct rpm database
version.  Sigh...

> In the meantime, I would be interested to know if anyone is
> successfully doing what the title of this message thread describes...
> Creating a Container with a different GuestOS than the HostOS, and if
> you could either provide a link or description to where the template
> script can be found (plus, as  Nathaniel Copa described anything
> special and extraordinary you had to do).

I have done it manually down through the years for many distros on
several distros under both OpenVZ and LXC with varying success.  It's
invariably a painful experience, often having to resort to the netinst
CD's or other precreated containers.  It's interesting that even OpenVZ
seems to have abandoned their original templating scheme and largely
gone with a pile of pre-built template tarballs on their website.  If we
can avoid recourse to that situation (and it has already been suggested
by more than one), we would be much better off.  I don't want to go down
the pre-build tarball road, myself.

> It would be done a lot faster except that this is being done in my spare time.
> Now, if someone who deals with LXC a lot wanted to hire me to do this
> and maybe more... :)

Honestly (and no offense - EVERYONE STOP LAUGHING) if you think you can
do it, have at it, knock yourself out.  But I won't support it.  In my
mind, it will end up being one great big unsupportable hairball of
functions and side effects and dependencies with too many fingers in the
pie for each distro's piece of the pie.  Too many years I've seen things
like that turn in to an exercise in autoerotism that is just not worth
the frustration.  If you want to take on the sole task of maintaining it
for every distro - more power to you.  I couldn't do it.  Who will
maintain it?

Keep it small and modularized into separate files with specific
functionality with very tightly controlled interfaces to reduce
interactions and side effects.  That way, I can take responsibility for
my files and you for yours.  When I pass my pieces along, I hope I've
commented things enough for someone else to take up the flame or flame
me if I failed.

If you can truly come up with a distro agnostic way of doing this
without requiring significant new support from the (arbitrary) guest
distros, my hat is off to you.  I'd love to see it.
> Tony

Apologies to anyone who feels I'm off base or too harsh here.

Regards,
Mike

> On Wed, Sep 4, 2013 at 10:52 AM, Natanael Copa <ncopa at alpinelinux.org> wrote:
> > On Wed, 04 Sep 2013 09:40:49 -0400
> > "Michael H. Warfield" <mhw at WittsEnd.com> wrote:
> >
> >> I do think it is an issue with the whole "distribution agnostic
> >> template" problem that may require some help from the distros or some
> >> innovative ideas of how we can bootstrap distros using distro agnostic
> >> tools (like stone knives and bear skins style install of the rootfs
> >> using nothing more than tar, gzip, gpg, and curl or wget).
> >
> > This would be very nice. I have not had success with any templates
> > except the debian on Alpine Linux. Debian works because we build a
> > debootstrap package. Ubuntu template did not work because it uses
> > 'arch' command which we don't have. (ok, should be trivial to implement
> > if we want it bad enough - and I haven't tested current git templates)
> >
> > However, the alpine template in current git should work on any distro.
> >
> > Here is what we do:
> > * download static apk-tools (package manager) and the package with the
> >   public keys used for package signature checking.
> >
> > * unpack the the package manager and public keys package with tar.  The
> >   package format is basically .tar.gz with some files in the beginning
> >   with metadata, so the .apk files can be extracted with tar -zx.
> >
> > * verify that the public keys are unmodified against a sha256 sum that
> >   is embedded in the template script.
> >
> > * verify that the static binary is unmodified using the public key and
> >   openssl. The apk-tools-static package includes a signature for the
> >   static binary.
> >
> > * use the verified static package manager to install a rootfs. The
> >   package manager will use the previously downloaded pub keys.
> >
> > This should work on any x86/x86_64 distro with tar, gzip, openssl and
> > wget.
> >
> >
> > -nc
> 

-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20130908/5594b2fb/attachment.pgp>