[lxc-devel] [distrobuilder/master] Add documentation
monstermunchkin on Github
lxc-bot at linuxcontainers.org
Tue Aug 27 13:55:01 UTC 2019
A non-text attachment was scrubbed...
Name: not available
Type: text/x-mailbox
Size: 310 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20190827/b25000b1/attachment.bin>
-------------- next part --------------
From 9f3dcfe8ad9b398a77dfc93f8b3725d3a1e0105d Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Wed, 21 Aug 2019 19:36:12 +0200
Subject: [PATCH 1/8] doc: Add generator documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/generators.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)
create mode 100644 doc/generators.md
diff --git a/doc/generators.md b/doc/generators.md
new file mode 100644
index 0000000..61eaa46
--- /dev/null
+++ b/doc/generators.md
@@ -0,0 +1,84 @@
+# Generators
+
+Generators are used to create, modify or remove files inside the rootfs.
+Available generators are
+
+* [cloud-init](#cloud-init)
+* [dump](#dump)
+* [hostname](#hostname)
+* [hosts](#hosts)
+* [remove](#remove)
+* [template](#template)
+* [upstart_tty](#upstart_tty)
+
+In the image definition yaml, they are listed under `files`.
+
+```yaml
+files:
+ - generator: <string> # which generator to use (required)
+ name: <string>
+ path: <string>
+ content: <string>
+ template:
+ properties: <map>
+ when: <array>
+ templated: <boolean>
+ architectures: <array> # filter
+ releases: <array> # filter
+ variants: <array> # filter
+```
+
+Filters can be applied to each entry in `files`.
+Valid filters are `architecture`, `release` and `variant`.
+See filters for more information.
+
+## cloud-init
+
+For LXC images, the generator disables cloud-init by disabling any cloud-init services, and creates the file `cloud-init.disable` which is checked by `cloud-init` on startup.
+
+For LXD images, the generator creates templates depending on the provided name.
+Valid names are `user-data`, `meta-data`, `vendor-data` and `network-config`.
+The default `path` if not defined otherwise is `/var/lib/cloud/seed/nocloud-net/<name>`.
+Setting `path`, `content` or `template.properties` will override the default values.
+
+## dump
+
+The `dump` generator writes the provided `content` to a file set in `path`.
+
+## hostname
+
+For LXC images, the hostname generator writes the LXC specific string `LXC_NAME` to the hostname file set in `path`.
+If the path doesn't exist, the generator does nothing.
+
+For LXD images, the generator creates a template for `path`.
+If the path doesn't exist, the generator does nothing.
+
+## hosts
+
+For LXC images, the generator adds the entry `127.0.0.1 LXC_NAME` to the hosts file set in `path`.
+
+For LXD images, the generator creates a template for the hosts file set in `path`, adding an entry for `127.0.0.1 {{ container.name }}`.
+
+## remove
+
+The generator removes the file set in `path` from the container's root filesystem.
+
+## template
+
+This generator creates a custom LXD template.
+The `name` field is used as the template's file name.
+The `path` defines the target file in the container's root filesystem.
+The `properties` key is a map of the template properties.
+
+The `when` key can be one or more of:
+
+* create (run at the time a new container is created from the image)
+* copy (run when a container is created from an existing one)
+* start (run every time the container is started)
+
+See [LXD image format](https://lxd.readthedocs.io/en/latest/image-handling/#image-format) for more.
+
+## upstart_tty
+
+This generator creates an upstart job which prevents certain TTYs from starting.
+The job script is written to `path`.
From 8f3e9020e1714d601cee24cdad211d7c81195074 Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Fri, 23 Aug 2019 19:09:43 +0200
Subject: [PATCH 2/8] doc: Add package documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/packages.md | 82 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
create mode 100644 doc/packages.md
diff --git a/doc/packages.md b/doc/packages.md
new file mode 100644
index 0000000..9521c74
--- /dev/null
+++ b/doc/packages.md
@@ -0,0 +1,82 @@
+# Package management
+
+Installing and removing packages can be done using the `packages` section.
+
+```yaml
+packages:
+ manager: <string> # required
+ update: <boolean>
+ cleanup: <boolean>
+ sets:
+ - packages:
+ - <string>
+ - ...
+ action: <string> # required
+ architectures: <array> # filter
+ releases: <array> # filter
+ variants: <array> # filter
+ - ...
+ repositories:
+ - name: <string>
+ url: <string>
+ type: <string>
+ key: <string>
+ architectures: <array> # filter
+ releases: <array> # filter
+ variants: <array> # filter
+ - ...
+
+```
+
+The `manager` keys specifies the package manager which is to be used.
+Valid package manager are:
+
+* apk
+* apt
+* dnf
+* egoportage (combination of portage and ego)
+* equo
+* opkg
+* pacman
+* portage
+* xbps
+* yum
+* zypper
+
+It's also possible to specify a custom package manager.
+This is useful if the desired package manager is not supported by distrobuilder.
+
+```yaml
+packages:
+ custom-manager: # required
+ clean: # required
+ command: <string>
+ flags: <array>
+ install: # required
+ command: <string>
+ flags: <array>
+ remove: # required
+ command: <string>
+ flags: <array>
+ refresh: # required
+ command: <string>
+ flags: <array>
+ update: # required
+ command: <string>
+ flags: <array>
+ flags: <array>
+ ...
+```
+
+If `update` is true, the package manager will update all installed packages.
+
+If `cleanup` is true, the package manager will run a cleanup operation which usually cleans up cached files.
+This depends on the package manager though and is not supported by all.
+
+A set contains a list of `packages`, an `action`, and optional filters.
+Here, `packages` is a list of packages which are to be installed or removed.
+The value of `action` must be either `install` or `remove`.
+
+`repositories` contains a list of additional repositories which are to be added.
+The `type` field is only needed if the package manager supports more than one repository manager.
+The `key` field is a GPG armored keyring which might be needed for verification.
From 01ddb8ed173700f7e01fbdd72d728b206efbf953 Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Fri, 23 Aug 2019 20:40:06 +0200
Subject: [PATCH 3/8] doc: Add actions documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/actions.md | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
create mode 100644 doc/actions.md
diff --git a/doc/actions.md b/doc/actions.md
new file mode 100644
index 0000000..03220db
--- /dev/null
+++ b/doc/actions.md
@@ -0,0 +1,33 @@
+# Actions
+
+```yaml
+actions:
+ - trigger: <string> # required
+ action: |-
+ #!/bin/bash
+ echo "Run me"
+ architectures: <array> # filter
+ releases: <array> # filter
+ variants: <array> # filter
+```
+
+Actions are scripts than are to be run after certain steps during the building process.
+Each action has two fields, `trigger` and `action`, as well as some filters.
+The `trigger` field describes the step after which the `action` is to be run.
+Valid triggers are:
+
+* `post-unpack`
+* `post-update`
+* `post-packages`
+* `post-files`
+
+The above list also shows the order in which the actions are processed.
+
+After the root filesystem has been unpacked, all `post-unpack` actions are run.
+
+After the package manager has updated all packages, (given that `packages.update` is `true`), all `post-update` all `post-packages` actions are run.
+After the package manager has installed the requested packages, all `post-packages` actions are run.
+For more on `packages`, see [packages](packages.md).
+
+And last, after the `files` section has been processed, all `post-files` actions are run.
+For more on `files`, see [generators](generators.md).
From bf04ed20dbb2af73c50f49c93ee8b574841ac66e Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Mon, 26 Aug 2019 15:42:27 +0200
Subject: [PATCH 4/8] doc: Add source documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/source.md | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 68 insertions(+)
create mode 100644 doc/source.md
diff --git a/doc/source.md b/doc/source.md
new file mode 100644
index 0000000..de68a6c
--- /dev/null
+++ b/doc/source.md
@@ -0,0 +1,68 @@
+# Source
+
+In order to create an image, a source must be defined.
+The source section is defined as follows:
+
+```yaml
+source:
+ downloader: <string> # required
+ url: <string>
+ keys: <array>
+ keyserver: <string>
+ variant: <string>
+ suite: <string>
+ same_as: <boolean>
+ apt_sources: <string>
+ skip_verification: <boolean>
+ early_packages: <array>
+```
+
+The `downloader` field defines a downloader which pulls a rootfs image which will be used as a starting point.
+It needs to be one of
+
+* alpinelinux-http
+* alt-http
+* archlinux-http
+* centos-http
+* debootstrap
+* docker-http
+* fedora-http
+* funtoo-http
+* gentoo-http
+* opensuse-http
+* openwrt-http
+* oraclelinux-http
+* sabayon-http
+* ubuntu-http
+* voidlinux-http
+
+The `url` field defines the URL or mirror of the rootfs image.
+Although this field is not required, most downloaders will need it.
+
+The `keys` field is a list of GPG keys.
+These keys can be listed as fingerprints or armored keys.
+The latter has the advantage of not having to rely on a keyserver to download the key from.
+The keys are used to verify the downloaded rootfs tarball if downloaded from a insecure source (http).
+
+The `keyserver` defaults to `hkps.pool.sks-keyservers.net` if none is provided.
+
+The `variant` field is only used in a few distributions and defaults to `default`.
+Here's a list downloaders and their possible variants:
+
+* `centos-http`: minimal, netinstall, LiveDVD
+* `debootstrap`: default, minbase, buildd, fakechroot
+* `ubuntu-http`: default, core
+* `voidlinux-http`: default, musl
+
+All other downloaders ignore this field.
+
+The `suite` field is only used by the `debootstrap` downloader.
+If set, debootstrap will use `suite` instead of `image.release` as its first positional argument.
+
+If the `same_as` field is set, distrobuilder creates a temporary symlink in `/usr/share/debootstrap/scripts` which points to the `same_as` file inside that directory.
+This can be used if you want to run `debootstrap foo` but `foo` is missing due to debootstrap not being up-to-date.
+
+If `skip_verification` is true, the source tarball is not verified.
+
+`early_packages` is a list of packages which is to be installed while the source is being downloaded.
+This is only used by the `debootstrap` downloader.
From dfca95fa217b7fdee4c5e99e0d26ec69c8e13b77 Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Mon, 26 Aug 2019 21:06:00 +0200
Subject: [PATCH 5/8] doc: Add image documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/image.md | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
create mode 100644 doc/image.md
diff --git a/doc/image.md b/doc/image.md
new file mode 100644
index 0000000..e4b63c1
--- /dev/null
+++ b/doc/image.md
@@ -0,0 +1,30 @@
+# Image
+
+The image section describes the image.
+
+```yaml
+image:
+ distribution: <string> # required
+ architecture: <string>
+ description: <string>
+ expiry: <string>
+ name: <string>
+ release: <string>
+ serial: <string>
+ variant: <string>
+```
+
+The fields `distribution`, `architecture`, `description` and `release` are self-explanatory.
+If `architecture` is not set, it defaults to the host's architecture.
+
+The `expiry` field describes the image expiry.
+The format is `\d+(s|m|h|d|w)` (seconds, minutes, hours, days, weeks), and defaults to 30 days (`30d`).
+It's also possible to define multiple such parts, e.g. `1h 30m 10s`.
+
+The `name` field is used in the LXD metadata as well as the output name for LXD unified tarballs.
+It defaults to `{{ image.distribution }}-{{ image.release }}-{{ image.architecture_mapped }}-{{ image.variant }}-{{ image.serial }}`.
+
+The `serial` field is the image's serial number.
+It can be anything and defaults to `YYYYmmdd_HHMM` (date format).
+
+The `variant` field can be anything and is used in the LXD metadata as well as for [filtering](filter.md).
From 48e61afc8276946cda5b082acee5308fb222833e Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Mon, 26 Aug 2019 21:51:06 +0200
Subject: [PATCH 6/8] doc: Add building documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/building.md | 91 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 91 insertions(+)
create mode 100644 doc/building.md
diff --git a/doc/building.md b/doc/building.md
new file mode 100644
index 0000000..4e8637b
--- /dev/null
+++ b/doc/building.md
@@ -0,0 +1,91 @@
+# Building images
+
+## Plain rootfs
+
+```shell
+$ distrobuilder build-dir --help
+Build plain rootfs
+
+Usage:
+ distrobuilder build-dir <filename|-> <target dir> [flags]
+
+Flags:
+ -h, --help help for build-dir
+
+Global Flags:
+ --cache-dir Cache directory
+ --cleanup Clean up cache directory (default true)
+ -o, --options Override options (list of key=value)
+ -t, --timeout Timeout in seconds
+```
+
+To build a plain rootfs, run `distrobuilder build-dir`.
+The command takes an image definition file and an output directory as positional arguments.
+Running `build-dir` is useful if one wants to build both LXC and LXD images.
+In that case one can simply run
+
+```shell
+distrobuilder build-dir def.yaml /path/to/rootfs
+distrobuilder pack-lxc def.yaml /path/to/rootfs /path/to/output
+distrobuilder pack-lxd def.yaml /path/to/rootfs /path/to/output
+```
+
+## LXC image
+
+```shell
+$ distrobuilder build-lxc --help
+Build LXC image from scratch
+
+Usage:
+ distrobuilder build-lxc <filename|-> [target dir] [flags]
+
+Flags:
+ -h, --help help for build-lxc
+
+Global Flags:
+ --cache-dir Cache directory
+ --cleanup Clean up cache directory (default true)
+ -o, --options Override options (list of key=value)
+ -t, --timeout Timeout in seconds
+```
+
+Running the `build-lxc` subcommand creates a LXC image.
+It outputs two files `rootfs.tar.xz` and `meta.tar.xz`.
+After building the image, the rootfs will be destroyed.
+
+The `pack-lxc` subcommand can be used to create an image from an existing rootfs.
+The rootfs won't be deleted afterwards.
+
+## LXD image
+
+```shell
+$ distrobuilder build-lxd --help
+Build LXD image from scratch
+
+Usage:
+ distrobuilder build-lxd <filename|-> [target dir] [--type=TYPE] [--compression=COMPRESSION] [flags]
+
+Flags:
+ --compression Type of compression to use (default "xz")
+ -h, --help help for build-lxd
+ --type Type of tarball to create (default "split")
+
+Global Flags:
+ --cache-dir Cache directory
+ --cleanup Clean up cache directory (default true)
+ -o, --options Override options (list of key=value)
+ -t, --timeout Timeout in seconds
+```
+
+Running the `build-lxd` subcommand creates a LXD image.
+If `--type=split`, it outputs two files `lxd.tar.xz` and `rootfs.squashfs`.
+This is the default.
+If `--type=unified`, a unified tarball named `<image.name>.tar.xz` is created.
+See the [image section](image.md) for more on the image name.
+
+If `--compression` is set, the tarballs will use the provided compression instead of `xz`.
+
+After building the image, the rootfs will be destroyed.
+
+The `pack-lxd` subcommand can be used to create an image from an existing rootfs.
+The rootfs won't be deleted afterwards.
From e99f96c4ad790664d6c8a2b917cb1d451371c720 Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Tue, 27 Aug 2019 12:23:22 +0200
Subject: [PATCH 7/8] doc: Add targets documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/targets.md | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
create mode 100644 doc/targets.md
diff --git a/doc/targets.md b/doc/targets.md
new file mode 100644
index 0000000..a563512
--- /dev/null
+++ b/doc/targets.md
@@ -0,0 +1,28 @@
+# Targets
+
+The target section is for target dependent files.
+Currently, the only supported target is LXC.
+
+```yaml
+targets:
+ lxc:
+ create-message: <string>
+ config:
+ - type: <string>
+ before: <uint>
+ after: <uint>
+ content: <string>
+ - ...
+```
+
+The `create-message` field is a string which is displayed after new LXC container has been created.
+This string is rendered using pongo2 and can include various fields from the definition file, e.g. `{{ image.description }}`.
+
+`config` is a list of container config options.
+The `type` must be `all`, `system` or `user`.
+
+The keys `before` and `after` are used for compatibility.
+Currently, the maximum value for compatability is 5.
+If your desired compatability level is 3 for example, you would use `before: 4` and `after: 2`.
+
+`content` describes the config which is to be written to the config file.
From 8d79740d412d9de68ecd1e83e237685843f8532f Mon Sep 17 00:00:00 2001
From: Thomas Hipp <thomas.hipp at canonical.com>
Date: Tue, 27 Aug 2019 15:21:50 +0200
Subject: [PATCH 8/8] doc: Add mappings documentation
Signed-off-by: Thomas Hipp <thomas.hipp at canonical.com>
---
doc/mappings.md | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
create mode 100644 doc/mappings.md
diff --git a/doc/mappings.md b/doc/mappings.md
new file mode 100644
index 0000000..c4b33e9
--- /dev/null
+++ b/doc/mappings.md
@@ -0,0 +1,40 @@
+# Mappings
+
+`mappings` describes an architecture mapping between the architectures from those used in LXD and those used by the distribution.
+These mappings are useful if you for example want to build a `x86_64` image but the source tarball contains `amd64` as its architecture.
+
+```yaml
+mappings:
+ architectures: <map>
+ architecture_map: <string>
+```
+
+It's possible to specify a custom map using the `architectures` field.
+Heres an example of a custom mapping:
+
+```yaml
+mappings:
+ architectures:
+ i686: i386
+ x86_64: amd64
+ armv7l: armhf
+ aarch64: arm64
+ ppc: powerpc
+ ppc64: powerpc64
+ ppc64le: ppc64el
+```
+
+The mapped architecture can be accessed via `Image.ArchitectureMapped` in the code or `image.architecture_mapped` in the definition file.
+
+There are some preset mappings which can be used in the `architecture_map` field.
+Those are:
+
+* alpinelinux
+* altlinux
+* archlinux
+* centos
+* debian
+* funtoo
+* gentoo
+* plamolinux
+* voidlinux
More information about the lxc-devel
mailing list