[Lxc-users] [lxc-devel] Working LXC templates?

Mon Sep 9 13:43:09 UTC 2013

Hello Michael,

Yes, I would really appreciate your comments on what I'm attempting to do.

Here is my thought process...

In general, I've found that the problems you describe are all too
common exactly because no one seems to have sat down and taken a close
look at the flow involved in creating a Container, installing the OS
from scratch like any other install instead of using pre-built
bootstrap file systems.

The problem has been exaccerbated by using templates which are
literally spaghetti code with little organization. When code like that
works, no one can modify the code because no one can reasonably see
what the consequences are and things will break without seeming rhyme
or reason.

This is why after looking at each template script, I'm choosing to
build on the Fedora template. Although it's broken several ways, I
admire its modularity and organization.so it should be a good
foundation.

I had considered something similar to what you're describing you are
building but decided against it because as I described eariler... If
you require a whole new set of tools (even cross platform) then it's
something else that needs to be maintained which I'd think should be
unnecessary. But, if someone decided to go down this path, then I
strongly considered using a standardized, cross platform packaging
system like Ruby.

But, as I said eariler, given a preference I would rather not require
installing new applications which largely duplicate what is already
available and likely installed in each distro when the problem is
mainly one of <configuration>, not missing functionality.

Without the bits of early code, this(following, below) is the
organization and structure of my proposed script...
I'm also still looking at the best cross-platform/cross-packaging way
to download the initial bootstrap packages, the fedora template
invokes pycurl. Still investigating whether it will work everywhere
and if something else (like wget if http is supported everywhere) is
more appropriate. But, after the bootstrap packages are downloaded,
the ContainerOS' native packaging manager can be used without
installing into the HostOS.

Note that putting everything in one large script also emphasizes the
re-usability of a number of modules across all distros, both HostOS
and Container creation.

<Global Variables>
Variables which apply to any and all Containers regardless of HostOS
or ContainerOS
</Global Variables>

<Default Distro Container Parameters>
Within this section there will be a section for each supported distro
Probably initially this may also be the main place to specify a
desired distro version for a Container but eventually making the
script interactive could be an eventual objective
These items will largely be determined by the content of the
<lxc-create> section
</Default Distro Container Parameters>

<HostOS "pre-Container" >
All the code in this section utilizes packages from the HostOS repos

<HostOS Identification>
This is the first part that is specific to the HostOS, so unless you
want to distribute different versions of this script for each distro,
this is where the HostOS must be identified, likely by reading
/etc/os-release
</HostOS Identification>

<Container creation>
    <lxc-create>
    Note that executing lxc-create is still in the context of the
HostOS, <using the HostOS packages> which means at this stage you
can't be for instance be using package managers not supported by the
HostOS. From what I can see lxc-create most importantly sets up the
chroot.

    This section also most importantly configures and downloads the
required packages into

    <Container Install>
    Execute a standard network install for the target distro or
something similar
    Any package managers specific to the target distro should run only
within this chroot environment
    </Container Install>

    </lxc-create>
</Container creation>

On Mon, Sep 9, 2013 at 5:10 AM, Michael H. Warfield <mhw at wittsend.com> wrote:
> On Mon, 2013-09-09 at 17:23 +0530, Shridhar Daithankar wrote:
>> On Monday, September 09, 2013 07:28:43 AM Michael H. Warfield wrote:
>> > In the git-stage current Fedora template, the entire problem is embodied
>> > in the "download_fedora" function starting around line 201...  The
>> > gotcha's are three commands around line 272 after we've identified and
>> > downloaded the initial release rpm.  We have this:
>> >
>> >     mkdir -p $INSTALL_ROOT/var/lib/rpm
>> >     rpm --root $INSTALL_ROOT  --initdb
>> >     rpm --root $INSTALL_ROOT -ivh ${INSTALL_ROOT}/${RELEASE_RPM}
>> >     $YUM install $PKG_LIST
>> >
>> > Ok...  Those are running in the host local run time environment.
>> > Obviously, if the host does not have a (compatible) version of rpm
>> > and/or yum in the host local environment, you're screwed.  That's the
>> > catch-22 situation and it's the same situation with zypper in the
>> > OpenSuse template.  That short space of code has to be recreated in a
>> > totally distro-agnostic manner so it runs on any distro to create our
>> > initial rootfs.  After that, we can do what ever distro (Fedora)
>> > specific commands we want by chrooting into the target container first
>> > (including rebuilding the rpm database to the target version).  That's
>> > only even needed if you don't already have a cached rootfs for that
>> > distro and version.
>
>> Another approach could be to popularize the container downloads by the distro.
>> If each distro. could add a .tar.gz for a working container of a given
>> release, one could just download and configure them, no?
>
>> then the lxc project upstream, could just list those links or include them in
>> the a separate tool, that just downloads and untars the same?
>
>> That would completely sidestep the bootstrapping one-distro.-on-another
>> problem.
>
> True, and that's been mentioned before in several different contexts.
> The problem is that we have to get the cooperation of ALL the distros we
> wish to support, which seems to be a bit of an intractable problem.  In
> Fedora, it would require someone to take that on as a project and be the
> maintainer.  It would also have to be architected into the build system
> so it would be automatically build when a release it cut.
>
> Since they already have the squashfs LiveOS system (which is 95% of what
> we need), I don't think it would be a major leap for them to add a
> parallel build to build an LXC LiveOS to live, say, in the same download
> directory.  In fact, if they fixed some of the deficiencies in the
> LiveOS image, we could work directly from the squashfs image that's
> already there.
>
> The problem is in getting someone interested (I'm not a Fedora
> maintainer) and getting them to do it.  It would probably have to be
> filled as an enhancement request and go through the months long vetting
> process at best.  We might have a better shot (in this case) of filing a
> bug report in bugzilla for the busted components of the LiveOS image and
> getting them to fix it.  Even there, though, it's likely impossible to
> get them to retroactively fix any of the previous images.
>
> I agree it would work, I just don't think we can depend on getting
> everyone else to agree to it and implement it in their distros.
>
> Considering the direction Fedora has been taking in recent decisions
> (removing sendmail / any MTA as well as the syslog server from the base
> install) makes me seriously question if they would even care.
>
>> --
>> Regards
>>  Shridhar
>
> Regards,
> Mike
> --
> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
>    /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
>    NIC whois: MHW9          | An optimist believes we live in the best of all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
>
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
> _______________________________________________
> Lxc-devel mailing list
> Lxc-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-devel
>