[lxc-devel] Last minute template addition - universal image based template
Stéphane Graber
stgraber at ubuntu.com
Fri Jan 10 21:56:59 UTC 2014
On Fri, Jan 10, 2014 at 04:16:18PM -0500, Leonid Isaev wrote:
> Hi,
>
> I have a few questions about this... sorry if they are uneducated.
>
> On Fri, 10 Jan 2014 15:10:14 -0500
> Stéphane Graber <stgraber at ubuntu.com> wrote:
>
> > Hey everyone,
> >
> > First of all, sorry for coming up with that so late in the 1.0
> > development cycle. I tried to convince myself for a long time that this
> > wasn't necessary but reality is that with unprivileged containers, we
> > need to start thinking about new ways to let our users create
> > containers.
> >
> > So briefly put, the idea is to introduce a new template called simply
> > enough "download" (I'm open to better name suggestions) which instead of
> > doing our usual magic of building a rootfs locally will instead download
> > a pre-built rootfs and some metadata from a central server.
>
> Current templates (after successfull execution) provide an up-to-date rootfs.
> With the prebuilt images this will be true only for "stable" distros like RHEL
> (and clones e.g. CentOS, Oracle), Debian and Ubuntu LTS. So how frequently
> these rootfs images are going to be updated?
My plan is for daily updates. With the client side expiry set to a maximum
of 30 days after which the client will attempt to update the on-disk
cache.
> >
> > My goal is to make the dependencies of that template minimal enough that
> > it should be able to run on every distribution that ships LXC, ideally,
> > including Android.
>
> Doesn't Fedora template do this already with its bootstrap code?
Fedora, ubuntu-cloud and cirros are three templates that are pretty
close to that at the moment indeed. However none of those will work with
just plain busybox (like we unfortunately have on Android).
> >
> >
> > Now as for the implementation, the plan is to bring up a new server on
> > the linuxcontainers.org domain which will sit at:
> > https://images.linuxcontainers.org
> >
> > The structure of that server will be something like
> > ├── images
> > │ ├── debian
> > │ │ └── wheezy
> > │ │ └── armhf
> > │ │ └── default
> > │ │ └── 2014-01-10_19:35
> > │ │ ├── meta.tar.gz
> > │ │ ├── meta.tar.gz.asc
> > │ │ ├── rootfs.tar.gz
> > │ │ └── rootfs.tar.gz.asc
> > │ └── ubuntu
> > │ └── 14.04
> > │ └── amd64
> > │ └── default
> > │ └── 2014-01-10_19:30
> > │ ├── meta.tar.gz
> > │ ├── meta.tar.gz.asc
> > │ ├── rootfs.tar.gz
> > │ └── rootfs.tar.gz.asc
> > └── meta
> > └── 1.0
> > ├── index-system
> > ├── index-system.asc
> > ├── index-user
> > └── index-user.asc
> >
> > So the filesystem structure is fairly logical and user friendly. Now as
> > for the details of the various files.
> >
> > rootfs.tar.gz is a tarball of only the root filesystem of the container
> > with the tarball starting at what will be the root of the container.
> >
> > meta.tar.gz is a tarball containing a bit of metadata which the template
> > script will need, that includes:
> > - config (mandatory; text file containing a minimal set of config
> > options to include in the container's config)
> > - fstab (mandatory; a standard fstab text file)
> > - expiry (mandatory; text file containing the Unix Epoch of the
> > recommended expiry date for the image)
> > - create-message (mandatory; text file which will be displayed to the
> > user post-create)
> > - *.<compat-level> (optional; any of the above may be overriden by
> > appending .<compat-level> to their name which will be
> > used instead of the original if the compat-level
> > matches that of the currently running LXC).
> >
> > The index-* files are standard CSV files with the following syntax:
> > - <distribution>;<version>;<architecture>;<variant>;<current build>;<url>
> >
> > index-system is for system containers, index-user is for unprivileged
> > containers. index-{system|user}.<compat-level> is also supported and
> > overrides the main index if found.
> >
> > All of those files are signed by the server's GPG key which will be
> > stored in-line in the script.
>
> So, the private key is also stored on the same server? In this case will the
> server key be signed by core developers?
The private key will be stored on an intermediate server, the tarball
builds will happen on build machines of the right architecture, the
result will then be pulled to the signing machine and the signed result
pushed out to the public web server.
You however raise a good point that having the public key in-line in the
script isn't ideal since a revocation of the key (should it somehow get
compromised) wouldn't prevent the tool from working.
Instead I'll change the tool to attempt to grab the key from a public
server using its keyid and only fallback to the built-in copy should
that fail (with appropriate warning).
>
> >
> >
> > The template itself will require the following arguments:
> > - distribution
> > - version
> > - architecture
>
> Some distros don't have versions... can we use a date instead, e.g. 20140105?
We could use "current" or "rolling" or something to that effect as the
name. The build date is already used as the build-id.
Using a date as the version is of course possible but would require the
user knowing that date which I guess is fairly unlikely.
> >
> > And the following will be optional:
> > - variant [default: default]
> > - server [default: https://images.linuxcontainers.org]
> > - pubkey [default: inline gpg key]
> > - no-validate [default: off]
> >
> > The goal for the template is to only depend on:
> > - POSIX shell
> > - wget
> > - tar
> > - gzip
> >
> > Which should be satisfiable even by busybox.
>
> Why not bzip2? It should reduce storage/bandwidth requirements and IIRC is
> supported by busybox...
Good point, old habits... I think I'll actually go with xz instead as
that seems to be kind of the standard nowadays and my understanding is
that it requires less CPU usage on the client side to unpack (and is
also supported by current busybox).
>
> >
> > However, the template will require --no-validate if gpg isn't also
> > installed on the system.
> >
> > The initial set of distros to be supported that way will be any distro
> > that can currently be built from an Ubuntu system (as that's what
> > linuxcontainers.org uses) and that uses the new common-config (currently
> > only ubuntu and ubuntu-cloud) setup (as I really don't want massive
> > config files that change all the time to be part of those images).
> >
> > The compat-level stuff mentioned above is there to allow us to change
> > the way the server works without breaking backward compatibility. It
> > also allows us to introduce new templates to the system only for
> > releases of LXC which support them.
> > The initial value will be "1" and I'm going to need a very very good
> > reason for bumping it (which is why I want all those distros to use
> > lxc.include for their config).
> >
> >
> > With all that said, I'll start working on the implementation of this and
> > hope to have the server working with ubuntu and debian images published
> > over the next few days.
> > If any other distro is interested (I very much hope they all will),
> > start by switching to lxc.include for your config, using something
> > similar to what ubuntu is doing, then get in touch with me.
> >
>
> Thanks,
> Leonid.
>
> --
> Leonid Isaev
> GnuPG key: 0x164B5A6D
> Fingerprint: C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140110/f25ab110/attachment.pgp>
More information about the lxc-devel
mailing list