[Lxc-users] [lxc-devel] Working LXC templates? EUREAKA! I think I've got it!

[Tony Su]

Regarding

LXC and the use of Linux Bridge devices for configured networking,
At least on openSUSE, LXC is not configured with any /etc/lxc/* by
default, and possibly because I also have libvirt configured to
support LXC (although that is not working on my system).

>From what I've seen though, I cannot see why configuring networking
should be in a general lxc configuration file, or even if it should
exist I see it as reaasonable to configure within a template(why not
point the build to a pre-existing virtual network complete with its
own configuration?).

In any case as I noted before, pointing the Container interface to a
pre-existing virtual Linux Bridge device instead of a physical
interface  in the template should be nearly trivial. If you wanted to
support bridge creation and configuration, that'd be a big project
outside the scope of what I'm describing.

********************************************
yum and chroot. Hmmmm. From what I can see, at the point you invoke
yum, nothing has yet been downloaded so any functionality you invoke
must exist in the HostOS. Despite running in the chroot environment, I
can't see how yum can work unless it's also present in the HostOS.
Maybe if I test this on a distro that doesn't support yum natively
I'll find different, but based purely on perusing the code I can't see
how it would work otherwise.

line 155
chroot ${rootfs_path} yum --releasever=${release} -y install fedora-release

********************************************
Requirement for GPG keys
Again, based at this point purely on perusing and not actually testing
the code on an appropriately set up system, unless the repo is
configured without keys or the retrieval utility is configured not to
require key verification, I don't see how you've avoided this
requirement, particularly in this case you're using "yum" which is a
repo client utility. This is aside from verifying the downloaded image
whose integrity could be verified easily by invoking and comparing
checksum values if desired. I'd recommend no verification, though
because it would add maintenance issues. A superior solution might be
a file transfer transport that automatically does checksum (like
torrent but that opens up potential issues since some ISPs block
torrent indiscriminitately).

Regards,
Tony
























On Tue, Sep 17, 2013 at 6:34 AM, Michael H. Warfield <mhw at wittsend.com> wrote:
> On Sun, 2013-09-15 at 16:17 -0700, Tony Su wrote:
>> Hello Michael,
>>
>> First a comment on problems with systemd you descrbe.
>> I probably have run into many of the things you itemized, but since my
>> time is usually focused on something I'm trying to use LXC and not LXC
>> itself, I usually just drop any further attempts and move on to find a
>> workaround(eg consoles) or use a different technology(x server issue).
>>
>> Regarding many of the issues you describe though, I wonder if they
>> couldn't be addressed with more strict enforcement of using namespaces
>> (and less often cgroups). I've read how namespaces are supposed to be
>> an extremely powerful means of isolating processes and yet I don't see
>> any obvious indications it's being done consistently... by either
>> prepending to standard process or service names (if the goal is to
>> easily identify the namespace) or using a random string (if the goal
>> is better security so exploits can't anticipate commonly used
>> namespaces).
>>
>> In fact, I think I see this namespace issue in various parts of the
>> template you created. If I understand what is happening, there are
>> numerous places where you create special nodes on the HostOS instead
>> of
>> (a) using the existing HostOS nodes but using namespaces to isolate
>> Container processes
>> (b) creating nodes entirely within the Container which would make the
>> Container entirely portable but lose the benefit perhaps of the better
>> ways nodes are created and mounted today(eg tmpfs in RAM).
>>
>> Diving more into your template code, I applaud your effort, it's
>> significant and no minor effort.
>>
>> As of this moment, I've mainly been perusing what I might call "HostOS
>> Container Pre-Install," the part which precedes the actual
>> installation and relies on components running in the HostOS only. This
>> would be your script approx lines 0-410.
>
>> 1. I like your method of identifying whether the OS is Fedora, and
>> additionally whether is ARM or not.
>
> That was an effort working on my Raspberry Pi's.
>
>> 2. It looks like you're configuring networking binding directly to
>> eth0. I would recommend instead supporting the use of Linux Bridge
>> devices, make declaration of a bridge device name as one of the early
>> Global Parameters, then if exists to bind to that device by name. Your
>> code to bind to the physical interface is less flexible but can be a
>> default option if no bridge device is specified.
>
> Original code.  I haven't really looked it it.  As far as bridges and
> all go, much of that is in the greater LXC setup and the default
> settings in /etc/lxc/default.conf and outside of our control.
>
>> 3. Interesting that you include an option for "nm controlled" yet at
>> least initially I don't see where your code might rely on this
>> setting.
>
> My code does not.  In the back of my mind, the setting for
> NetworkMangler control is something that is used internally to the
> container and we're merely setting a default value.
>
>> 4. mknod I'll have to take a closer look whether and why you appear to
>> be setting up various consoles, some /dev/ nodes, an explicit console
>> path, more. I've generally been under the impression that a full
>> install automatically creates these. Peeking a bit ahead of where I've
>> been reading your script, I notice your install method uses a
>> pre-built squashfs image, perhaps these are a special requirement
>> because your chosen squashfs image doesn't include these by default or
>> requires those nodes to already exist?
>
>> 5. Your use of yum will work in the RH family plus various others like
>> openSUSE but I don't think it's native to distros like the Debian
>> family. IMO there is no special benefit to using a package manager
>> specific to the HostOS to download bootstrap images and packages, they
>> aren't too relevant to the overall apps running in the HostOS and I
>> think we should avoid installing non-native packages in any OS. For
>> that reason, I've been looking at pycurl, curl and wget which are
>> generic apps common to all distros which can accomplish the simple
>> task of retrieving the bootstrap objects. (See the template I included
>> as an attachment which uses pycurl and finds fedora repos rather than
>> installing a pre-built img)
>
> Because yum (and rpm) are no ONLY and EVER executed in a chrooted
> environment, they are executing in a Fedora context and independent of
> the host environment.  That's the whole point, it never execute the
> distro-dependent components in the host environment and only execute
> distro-agnostic (independent) components in the host environment.
>
>> A small FYI -
>> Although the Fedora template distributed with openSUSE which I've
>> included as an attachment to this message <does not work as is> it
>> might be useful to see a different way of obtaining and installing
>> Fedora bootstrap packages so I've included it as an attachment to this
>> message.
>
> I've said this before.  Take it to the bank.  The template that is
> distributed with OpenSuse is OUT OF DATE AND WILL NOT WORK!  In fact,
> the entire LXC package that is distributed with OpenSuse is out of date,
> but that is beyond our control.
>
> The template I sent out is also very preliminary and will be heavily
> revised before being submitted as a patch (now, after the LinuxPlumbers
> conference).  It may not even get into the 1.0.0 release at this point
> as we're already seeing our first Alpha tags and pulls.
>
>> I've made the two following modifications
>> 1. line 33 - added "release=18"
>> The comments in this script describe passing the release number as an
>> option to "lxc-create" but is not supported in openSUSE. Despite
>> unable to pass as a command line option, passing within the template
>> with this line works. BTW - If no release is specified, Fedora
>> defaults to earliest release in the repos(which is 14) rather than
>> latest.
>> 2. Line 153 - The template hardcodes the RELEASE_URL string which is
>> created by appending a hardcoded string to the MIRROR_URL string, but
>> it appears that Fedora restructured their repos since this template
>> was created. Now, an "/f/" has to be inserted into the RELEASE_URL
>> (initial letter of the word "fedora").
>
> Fedora, after a certain release, changed their "Packages" directory to
> include a first letter of the package as a subdirectory.  So
> "fedora-release" is in "Packages/f/fedora-release-*" and yum is in
> "Packages/y/yum-*".  That's the Fedora convention now as of a certain
> rev of Fedora.  There's an if check in there for it.
>
>> Additional - Especially when connecting to repos of a Distro different
>> than the HostOS, GPG authentication keys are not yet installed. Have
>> been investigating whether it's possible to simply download ahead of
>> time and install into the default Key Ring or something more is
>> required. If this approach is feasable, then this needs to be added
>> early to the template script, but maybe a better method is for the
>> User to be prompted for an interactive answer to confirm key
>> installation.
>
> Actually, this should now be largely resolved by using the LiveOS image
> and minimizing the use of --nogpgcheck to yum, but I'll verify that.
> There is a catch-22 of verifying the LiveOS image but I'm not sure I
> have a solution to that one.
>
>> Tony
>
>> On Sat, Sep 14, 2013 at 3:35 PM, Tony Su <tonysu at su-networking.com> wrote:
>> > Cool.
>> >
>> > I'll block some significant time to look at what you built over the next 3
>> > days.
>> >
>> > Tony
>
> Later!
>
> Regards,
> Mike
> --
> Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
>    /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
>    NIC whois: MHW9          | An optimist believes we live in the best of all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!