[lxc-devel] Last minute template addition - universal image based template
Stéphane Graber
stgraber at ubuntu.com
Tue Jan 14 02:28:53 UTC 2014
On Mon, Jan 13, 2014 at 08:56:23PM -0500, Stéphane Graber wrote:
> On Mon, Jan 13, 2014 at 08:04:56PM -0500, Dwight Engen wrote:
> > On Fri, 10 Jan 2014 15:10:14 -0500
> > Stéphane Graber <stgraber at ubuntu.com> wrote:
> >
> > > Hey everyone,
> > >
> > > First of all, sorry for coming up with that so late in the 1.0
> > > development cycle. I tried to convince myself for a long time that
> > > this wasn't necessary but reality is that with unprivileged
> > > containers, we need to start thinking about new ways to let our users
> > > create containers.
> >
> > Interesting idea, and I can see how it might be helpful. I converted
> > the Oracle template to common.conf style and tried this out a bit. Some
> > comments inline along with some questions.
>
> Cool!
>
> >
> > > So briefly put, the idea is to introduce a new template called simply
> > > enough "download" (I'm open to better name suggestions) which instead
> > > of doing our usual magic of building a rootfs locally will instead
> > > download a pre-built rootfs and some metadata from a central server.
> > >
> > > My goal is to make the dependencies of that template minimal enough
> > > that it should be able to run on every distribution that ships LXC,
> > > ideally, including Android.
> > >
> > >
> > > Now as for the implementation, the plan is to bring up a new server on
> > > the linuxcontainers.org domain which will sit at:
> > > https://images.linuxcontainers.org
> > >
> > > The structure of that server will be something like
> > > ├── images
> > > │ ├── debian
> > > │ │ └── wheezy
> > > │ │ └── armhf
> > > │ │ └── default
> > > │ │ └── 2014-01-10_19:35
> > > │ │ ├── meta.tar.gz
> > > │ │ ├── meta.tar.gz.asc
> > > │ │ ├── rootfs.tar.gz
> > > │ │ └── rootfs.tar.gz.asc
> > > │ └── ubuntu
> > > │ └── 14.04
> > > │ └── amd64
> > > │ └── default
> > > │ └── 2014-01-10_19:30
> > > │ ├── meta.tar.gz
> > > │ ├── meta.tar.gz.asc
> > > │ ├── rootfs.tar.gz
> > > │ └── rootfs.tar.gz.asc
> > > └── meta
> > > └── 1.0
> > > ├── index-system
> > > ├── index-system.asc
> > > ├── index-user
> > > └── index-user.asc
> > >
> > > So the filesystem structure is fairly logical and user friendly. Now
> > > as for the details of the various files.
> > >
> > > rootfs.tar.gz is a tarball of only the root filesystem of the
> > > container with the tarball starting at what will be the root of the
> > > container.
> > >
> > > meta.tar.gz is a tarball containing a bit of metadata which the
> > > template script will need, that includes:
> > > - config (mandatory; text file containing a minimal set of config
> > > options to include in the container's config)
> > > - fstab (mandatory; a standard fstab text file)
> > > - expiry (mandatory; text file containing the Unix Epoch of the
> > > recommended expiry date for the image)
> >
> > These two didn't actually seem to be mandatory, at least not yet :)
>
> Yeah, I did a few last-minute changes.
>
> fstab is no longer required (only copied over if it's there).
>
> If the expiry file is messing, it's simply assumed that the container
> won't expire.
>
> >
> > > - create-message (mandatory; text file which will be displayed to the
> > > user post-create)
> > > - *.<compat-level> (optional; any of the above may be overriden by
> > > appending .<compat-level> to their name which
> > > will be used instead of the original if the compat-level
> > > matches that of the currently running LXC).
> > >
> > > The index-* files are standard CSV files with the following syntax:
> > > - <distribution>;<version>;<architecture>;<variant>;<current
> > > build>;<url>
> >
> > I grabbed the index-system that is up on images.linuxcontainers.org,
> > and in that file shouldn't the url have a "variant" "default" path
> > component in it to be consistent with the above tree? ie:
> > /images/ubuntu/trusty/amd64/2014-01-10_23:15 ->
> > /images/ubuntu/trusty/amd64/default/2014-01-10_23:15
>
> Yeah, what you saw was before the switch to the actual server code
> (which occured around 3 minutes ago :)), URLs now contain the variant.
>
> >
> > > index-system is for system containers, index-user is for unprivileged
> > > containers. index-{system|user}.<compat-level> is also supported and
> > > overrides the main index if found.
> > >
> > > All of those files are signed by the server's GPG key which will be
> > > stored in-line in the script.
> >
> > Patch included at the bottom to get the script to not even attempt to
> > download the signatures when --no-validate is passed since on my test
> > http server I didn't bother with any of the signature files.
>
> Looks good, I'll push it in a bit.
>
> >
> > > The template itself will require the following arguments:
> > > - distribution
> > > - version
> > > - architecture
> > >
> > > And the following will be optional:
> > > - variant [default: default]
> > > - server [default: https://images.linuxcontainers.org]
> > > - pubkey [default: inline gpg key]
> > > - no-validate [default: off]
> > >
> > > The goal for the template is to only depend on:
> > > - POSIX shell
> > > - wget
> > > - tar
> > > - gzip
> >
> > It also uses recursive chown, which busybox does seem to have but
> > thought I'd note it.
> >
> > > Which should be satisfiable even by busybox.
> > >
> > > However, the template will require --no-validate if gpg isn't also
> > > installed on the system.
> > >
> > > The initial set of distros to be supported that way will be any distro
> > > that can currently be built from an Ubuntu system (as that's what
> > > linuxcontainers.org uses) and that uses the new common-config
> > > (currently only ubuntu and ubuntu-cloud) setup (as I really don't
> > > want massive config files that change all the time to be part of
> > > those images).
> >
> > Just trying to make sure I understand here: The plan is to
> > automatically build a rootfs for any distro that lxc can currently
> > build a rootfs via lxc-create on ubuntu host? Thats probably ~40 or so
> > just for Oracle Linux if we did all the versions (I have an sh+expect
> > test scripts that I use to make sure they all work), but maybe we'd
> > only want the latest of each (ie. 4, 5, 6) which cuts it way down to
> > only 3.
Oh, also forgot to mention. I currently only have Ubuntu containers in
my Jenkins' build slaves but once it starts becoming a problem, adding
other distributions should be trivial so long as the distro doesn't mind
running on an Ubuntu host and someone can provide me with an initial
rootfs tarball (chicken and egg problem :)).
>
> So currently my plan for Ubuntu and Debian is to build all
> supported/devel releases (5 for Ubuntu, 3 for Debian) on i386, amd64,
> armhf and armel (given that the specific release supports the
> architecture).
>
> That's 20 builds for Ubuntu and 12 for Debian.
> I'm not overly concerned about having a lot of i386/amd64 builds as I
> can easily add more Jenkins slaves for those architectures. armel/armhf
> is a bit trickier as I only have one good armhf system so if we start
> having say 50 images to build a day on ARM, it's going to become a
> problem.
>
> The current schedule is of one build per day. This can easily be changed
> at the template level and possibly also per architecture and release
> (not a Jenkins guru but I suspect we can hack something to make this
> possible).
>
> Now for Oracle, I guess it depends what's most useful for your users.
>
> Is there actually still a demand for 4.x? I thought at least RHEL 4.x
> went out of support last year.
>
> In any case, I suspect that starting with the latest of each supported
> release on i386 and amd64 is probably a good start.
> We can then expand that to include popular minor releases, possibly the
> last few 6.x?
>
> >
> > One question I have on this is that the rootfs (at least that the
> > Oracle template makes, not sure about others) isn't quite right since
> > the host name will have already been injected into config files (for
> > example /etc/hosts), but lxc-download wants to template them itself.
> > Should we add a switch to the distro template so it knows when its
> > being called to create a rootfs for lxc-download to put LXC_NAME in
> > there instead the --name it was passed?
>
> So currently my build script (Jenkins job) will call the template with:
> - --rootfs=/build-lxc/container/LXC_NAME/rootfs
> - --path=/build-lxc/container/LXC_NAME
> - --name=LXC_NAME
>
> Which should have the template generate a container called "LXC_NAME"
> which is the value the client template looks for when templating
> conffiles.
>
> >
> > > The compat-level stuff mentioned above is there to allow us to change
> > > the way the server works without breaking backward compatibility. It
> > > also allows us to introduce new templates to the system only for
> > > releases of LXC which support them.
> > > The initial value will be "1" and I'm going to need a very very good
> > > reason for bumping it (which is why I want all those distros to use
> > > lxc.include for their config).
> > >
> > >
> > > With all that said, I'll start working on the implementation of this
> > > and hope to have the server working with ubuntu and debian images
> > > published over the next few days.
> > > If any other distro is interested (I very much hope they all will),
> > > start by switching to lxc.include for your config, using something
> > > similar to what ubuntu is doing, then get in touch with me.
> >
> > I have not tried from a normal user account yet. Is it basically that
> > you expect the template to be run by a normal user, but be running in a
> > userns? Can you describe how mapped_uid is supposed to be set? Thanks!
>
> Yeah, lxc-create basically does all the magic we need there.
> When run as a user, it'll run the template in a userns mapped with the
> container's range.
>
> $mapped_uid appears to be set to the uid in the userns which maps back
> to the user's own uid outside of it and makes it possible to chown files
> back to the actual user.
>
> >
> > -->8 8<--
> >
> > From 91e4ac620496f896eacd96a44eed25183267d496 Mon Sep 17 00:00:00 2001
> > From: Dwight Engen <dwight.engen at oracle.com>
> > Date: Mon, 13 Jan 2014 19:54:02 -0500
> > Subject: [PATCH] download template: don't download signatures when
> > --no-validate given
> >
> > - show full path to failed download location
> >
> > - change test to -f in case meta.tar.xz:templates has a blank line it
> > won't attempt to sed a directory
> >
> > Signed-off-by: Dwight Engen <dwight.engen at oracle.com>
> > ---
> > templates/lxc-download.in | 20 ++++++++++++++------
> > 1 file changed, 14 insertions(+), 6 deletions(-)
> >
> > diff --git a/templates/lxc-download.in b/templates/lxc-download.in
> > index cab6cbc..fa54686 100644
> > --- a/templates/lxc-download.in
> > +++ b/templates/lxc-download.in
> > @@ -60,7 +60,7 @@ download_file() {
> > if [ "$3" = "noexit" ]; then
> > return 1
> > else
> > - echo "ERROR: Failed to download $1" 1>&2
> > + echo "ERROR: Failed to download http://${DOWNLOAD_SERVER}/$1" 1>&2
> > exit 1
> > fi
> > elif [ "$DOWNLOAD_SHOW_HTTP_WARNING" = "true" ]; then
> > @@ -72,6 +72,14 @@ download_file() {
> > fi
> > }
> >
> > +download_sig() {
> > + download_file $1 $2 noexit
> > + if [ "$DOWNLOAD_VALIDATE" = "true" ]; then
> > + echo "ERROR: Failed to download http://${DOWNLOAD_SERVER}/$1" 1>&2
> > + exit 1
> > + fi
> > +}
> > +
> > gpg_setup() {
> > if [ "$DOWNLOAD_VALIDATE" = "false" ]; then
> > return
> > @@ -263,8 +271,8 @@ if [ "$DOWNLOAD_USE_CACHE" = "false" ]; then
> > ! download_file ${DOWNLOAD_INDEX_PATH}.${DOWNLOAD_COMPAT_LEVEL}.asc \
> > ${DOWNLOAD_TEMP}/index.asc noexit; then
> > download_file ${DOWNLOAD_INDEX_PATH} ${DOWNLOAD_TEMP}/index normal
> > - download_file ${DOWNLOAD_INDEX_PATH}.asc \
> > - ${DOWNLOAD_TEMP}/index.asc normal
> > + download_sig ${DOWNLOAD_INDEX_PATH}.asc \
> > + ${DOWNLOAD_TEMP}/index.asc
> > fi
> >
> > gpg_validate ${DOWNLOAD_TEMP}/index.asc
> > @@ -298,14 +306,14 @@ if [ "$DOWNLOAD_USE_CACHE" = "false" ]; then
> > echo "Downloading the rootfs"
> > download_file $DOWNLOAD_URL/rootfs.tar.xz \
> > ${DOWNLOAD_TEMP}/rootfs.tar.xz normal
> > - download_file $DOWNLOAD_URL/rootfs.tar.xz.asc \
> > + download_sig $DOWNLOAD_URL/rootfs.tar.xz.asc \
> > ${DOWNLOAD_TEMP}/rootfs.tar.xz.asc normal
> > gpg_validate ${DOWNLOAD_TEMP}/rootfs.tar.xz.asc
> >
> > echo "Downloading the metadata"
> > download_file $DOWNLOAD_URL/meta.tar.xz \
> > ${DOWNLOAD_TEMP}/meta.tar.xz normal
> > - download_file $DOWNLOAD_URL/meta.tar.xz.asc \
> > + download_sig $DOWNLOAD_URL/meta.tar.xz.asc \
> > ${DOWNLOAD_TEMP}/meta.tar.xz.asc normal
> > gpg_validate ${DOWNLOAD_TEMP}/meta.tar.xz.asc
> >
> > @@ -393,7 +401,7 @@ fi
> >
> > # Replace variables in all templates
> > for file in $TEMPLATE_FILES; do
> > - [ ! -e "$file" ] && continue
> > + [ ! -f "$file" ] && continue
> >
> > sed -i "s#LXC_NAME#$LXC_NAME#g" $file
> > sed -i "s#LXC_PATH#$LXC_PATH#g" $file
> > --
> > 1.8.3.1
> >
>
> --
> Stéphane Graber
> Ubuntu developer
> http://www.ubuntu.com
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140113/6fb91bc3/attachment-0001.pgp>
More information about the lxc-devel
mailing list