[lxc-devel] [lxd/master] Add hugepages support for VMs

stgraber on Github lxc-bot at linuxcontainers.org
Tue Nov 26 21:31:40 UTC 2019


A non-text attachment was scrubbed...
Name: not available
Type: text/x-mailbox
Size: 301 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20191126/3ed7a9b4/attachment-0001.bin>
-------------- next part --------------
From 32d7e65ae53c56224ae90c6ac1c61e214c6f2ffb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= <stgraber at ubuntu.com>
Date: Tue, 26 Nov 2019 16:16:34 -0500
Subject: [PATCH 1/5] lxd/vm: Add limits.memory.hugepages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber <stgraber at ubuntu.com>
---
 lxd/vm_qemu.go | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lxd/vm_qemu.go b/lxd/vm_qemu.go
index 2ecc713bf6..c68ca7e95e 100644
--- a/lxd/vm_qemu.go
+++ b/lxd/vm_qemu.go
@@ -563,6 +563,9 @@ func (vm *vmQemu) Start(stateful bool) error {
 		"-readconfig", confFile,
 		"-pidfile", vm.pidFilePath(),
 	}
+	if shared.IsTrue(vm.expandedConfig["limits.memory.hugepages"]) {
+		args = append(args, "-mem-path", "/dev/hugepages/", "-mem-prealloc")
+	}
 
 	if vm.expandedConfig["raw.qemu"] != "" {
 		fields := strings.Split(vm.expandedConfig["raw.qemu"], " ")
@@ -1079,13 +1082,11 @@ func (vm *vmQemu) addMemoryConfig(sb *strings.Builder) error {
 		return fmt.Errorf("limits.memory invalid: %v", err)
 	}
 
-	memKB := memSizeBytes / 1000
-
 	sb.WriteString(fmt.Sprintf(`
 # Memory
 [memory]
-size = "%dK"
-`, memKB))
+size = "%dB"
+`, memSizeBytes))
 
 	return nil
 }

From 8e3b9dc8a85507fe355a5754ca1e334f858cce0f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= <stgraber at ubuntu.com>
Date: Tue, 26 Nov 2019 16:16:43 -0500
Subject: [PATCH 2/5] shared: Add limits.memory.hugepages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber <stgraber at ubuntu.com>
---
 shared/instance.go | 1 +
 1 file changed, 1 insertion(+)

diff --git a/shared/instance.go b/shared/instance.go
index a1c41dc996..985ebd4fce 100644
--- a/shared/instance.go
+++ b/shared/instance.go
@@ -266,6 +266,7 @@ var KnownInstanceConfigKeys = map[string]func(value string) error{
 	},
 	"limits.memory.swap":          IsBool,
 	"limits.memory.swap.priority": IsPriority,
+	"limits.memory.hugepages":     IsBool,
 
 	"limits.network.priority": IsPriority,
 

From ef004b91ab40cd7a72b804b15748cba0990af945 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= <stgraber at ubuntu.com>
Date: Tue, 26 Nov 2019 16:16:48 -0500
Subject: [PATCH 3/5] doc: Add limits.memory.hugepages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber <stgraber at ubuntu.com>
---
 doc/instances.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/instances.md b/doc/instances.md
index d1f74cf112..d7486abe27 100644
--- a/doc/instances.md
+++ b/doc/instances.md
@@ -49,6 +49,7 @@ limits.disk.priority                            | integer   | 5 (medium)
 limits.kernel.\*                                | string    | -                 | no            | This limits kernel resources per instance (e.g. number of open files)
 limits.memory                                   | string    | - (all)           | yes           | Percentage of the host's memory or fixed value in bytes (various suffixes supported, see below)
 limits.memory.enforce                           | string    | hard              | yes           | If hard, instance can't exceed its memory limit. If soft, the instance can exceed its memory limit when extra host memory is available
+limits.memory.hugepages                         | boolean   | false             | no            | Controls whether to back the instance using hugepages rather than regular system memory
 limits.memory.swap                              | boolean   | true              | yes           | Whether to allow some of the instance's memory to be swapped out to disk
 limits.memory.swap.priority                     | integer   | 10 (maximum)      | yes           | The higher this is set, the least likely the instance is to be swapped to disk (integer between 0 and 10)
 limits.network.priority                         | integer   | 0 (minimum)       | yes           | When under load, how much priority to give to the instance's network requests (integer between 0 and 10)

From b946f0f316ee626d54cb040005362482024c5e0d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= <stgraber at ubuntu.com>
Date: Tue, 26 Nov 2019 16:16:55 -0500
Subject: [PATCH 4/5] scripts/bash: Add limits.memory.hugepages
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber <stgraber at ubuntu.com>
---
 scripts/bash/lxd-client | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/bash/lxd-client b/scripts/bash/lxd-client
index 805a30d444..b437c10524 100644
--- a/scripts/bash/lxd-client
+++ b/scripts/bash/lxd-client
@@ -84,7 +84,7 @@ _have lxc && {
       boot.host_shutdown_timeout environment. \
       limits.cpu limits.cpu.allowance limits.cpu.priority \
       limits.disk.priority limits.memory limits.memory.enforce \
-      limits.kernel \
+      limits.memory.hugepages limits.kernel \
       limits.memory.swap limits.memory.swap.priority limits.network.priority \
       limits.processes linux.kernel_modules migration.incremental.memory \
       migration.incremental.memory.goal nvidia.runtime \

From a6698af065bd018e01b57ec517521af4853c5c5b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?St=C3=A9phane=20Graber?= <stgraber at ubuntu.com>
Date: Tue, 26 Nov 2019 16:30:03 -0500
Subject: [PATCH 5/5] doc/instances: Indicate VM support when applicable
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Signed-off-by: Stéphane Graber <stgraber at ubuntu.com>
---
 doc/instances.md | 150 +++++++++++++++++++++++------------------------
 1 file changed, 74 insertions(+), 76 deletions(-)

diff --git a/doc/instances.md b/doc/instances.md
index d7486abe27..9ef017d60a 100644
--- a/doc/instances.md
+++ b/doc/instances.md
@@ -34,64 +34,64 @@ currently supported:
 
 The currently supported keys are:
 
-Key                                             | Type      | Default           | Live update   | Description
-:--                                             | :---      | :------           | :----------   | :----------
-boot.autostart                                  | boolean   | -                 | n/a           | Always start the instance when LXD starts (if not set, restore last state)
-boot.autostart.delay                            | integer   | 0                 | n/a           | Number of seconds to wait after the instance started before starting the next one
-boot.autostart.priority                         | integer   | 0                 | n/a           | What order to start the instances in (starting with highest)
-boot.host\_shutdown\_timeout                    | integer   | 30                | yes           | Seconds to wait for instance to shutdown before it is force stopped
-boot.stop.priority                              | integer   | 0                 | n/a           | What order to shutdown the instances (starting with highest)
-environment.\*                                  | string    | -                 | yes (exec)    | key/value environment variables to export to the instance and set on exec
-limits.cpu                                      | string    | - (all)           | yes           | Number or range of CPUs to expose to the instance
-limits.cpu.allowance                            | string    | 100%              | yes           | How much of the CPU can be used. Can be a percentage (e.g. 50%) for a soft limit or hard a chunk of time (25ms/100ms)
-limits.cpu.priority                             | integer   | 10 (maximum)      | yes           | CPU scheduling priority compared to other instances sharing the same CPUs (overcommit) (integer between 0 and 10)
-limits.disk.priority                            | integer   | 5 (medium)        | yes           | When under load, how much priority to give to the instance's I/O requests (integer between 0 and 10)
-limits.kernel.\*                                | string    | -                 | no            | This limits kernel resources per instance (e.g. number of open files)
-limits.memory                                   | string    | - (all)           | yes           | Percentage of the host's memory or fixed value in bytes (various suffixes supported, see below)
-limits.memory.enforce                           | string    | hard              | yes           | If hard, instance can't exceed its memory limit. If soft, the instance can exceed its memory limit when extra host memory is available
-limits.memory.hugepages                         | boolean   | false             | no            | Controls whether to back the instance using hugepages rather than regular system memory
-limits.memory.swap                              | boolean   | true              | yes           | Whether to allow some of the instance's memory to be swapped out to disk
-limits.memory.swap.priority                     | integer   | 10 (maximum)      | yes           | The higher this is set, the least likely the instance is to be swapped to disk (integer between 0 and 10)
-limits.network.priority                         | integer   | 0 (minimum)       | yes           | When under load, how much priority to give to the instance's network requests (integer between 0 and 10)
-limits.processes                                | integer   | - (max)           | yes           | Maximum number of processes that can run in the instance
-linux.kernel\_modules                           | string    | -                 | yes           | Comma separated list of kernel modules to load before starting the instance
-migration.incremental.memory                    | boolean   | false             | yes           | Incremental memory transfer of the instance's memory to reduce downtime
-migration.incremental.memory.goal               | integer   | 70                | yes           | Percentage of memory to have in sync before stopping the instance
-migration.incremental.memory.iterations         | integer   | 10                | yes           | Maximum number of transfer operations to go through before stopping the instance
-nvidia.driver.capabilities                      | string    | compute,utility   | no            | What driver capabilities the instance needs (sets libnvidia-container NVIDIA\_DRIVER\_CAPABILITIES)
-nvidia.runtime                                  | boolean   | false             | no            | Pass the host NVIDIA and CUDA runtime libraries into the instance
-nvidia.require.cuda                             | string    | -                 | no            | Version expression for the required CUDA version (sets libnvidia-container NVIDIA\_REQUIRE\_CUDA)
-nvidia.require.driver                           | string    | -                 | no            | Version expression for the required driver version (sets libnvidia-container NVIDIA\_REQUIRE\_DRIVER)
-raw.apparmor                                    | blob      | -                 | yes           | Apparmor profile entries to be appended to the generated profile
-raw.idmap                                       | blob      | -                 | no            | Raw idmap configuration (e.g. "both 1000 1000")
-raw.lxc                                         | blob      | -                 | no            | Raw LXC configuration to be appended to the generated one
-raw.qemu                                        | blob      | -                 | no            | Raw Qemu configuration to be appended to the generated command line
-raw.seccomp                                     | blob      | -                 | no            | Raw Seccomp configuration
-security.devlxd                                 | boolean   | true              | no            | Controls the presence of /dev/lxd in the instance
-security.devlxd.images                          | boolean   | false             | no            | Controls the availability of the /1.0/images API over devlxd
-security.idmap.base                             | integer   | -                 | no            | The base host ID to use for the allocation (overrides auto-detection)
-security.idmap.isolated                         | boolean   | false             | no            | Use an idmap for this instance that is unique among instances with isolated set
-security.idmap.size                             | integer   | -                 | no            | The size of the idmap to use
-security.nesting                                | boolean   | false             | yes           | Support running lxd (nested) inside the instance
-security.privileged                             | boolean   | false             | no            | Runs the instance in privileged mode
-security.protection.delete                      | boolean   | false             | yes           | Prevents the instance from being deleted
-security.protection.shift                       | boolean   | false             | yes           | Prevents the instance's filesystem from being uid/gid shifted on startup
-security.secureboot                             | boolean   | true              | no            | Controls whether UEFI secure boot is enabled with the default Microsoft keys
-security.syscalls.blacklist                     | string    | -                 | no            | A '\n' separated list of syscalls to blacklist
-security.syscalls.blacklist\_compat             | boolean   | false             | no            | On x86\_64 this enables blocking of compat\_\* syscalls, it is a no-op on other arches
-security.syscalls.blacklist\_default            | boolean   | true              | no            | Enables the default syscall blacklist
-security.syscalls.intercept.mknod               | boolean   | false             | no            | Handles the `mknod` and `mknodat` system calls (allows creation of a limited subset of char/block devices)
-security.syscalls.intercept.mount               | boolean   | false             | no            | Handles the `mount` system call
-security.syscalls.intercept.mount.allowed       | string    | -                 | yes           | Specify a comma-separated list of filesystems that are safe to mount for processes inside the instance
-security.syscalls.intercept.mount.fuse          | string    | -                 | yes           | Whether to mount shiftfs on top of filesystems handled through mount syscall interception
-security.syscalls.intercept.mount.shift         | boolean   | false             | yes           | Whether to redirect mounts of a given filesystem to their fuse implemenation (e.g. ext4=fuse2fs)
-security.syscalls.intercept.setxattr            | boolean   | false             | no            | Handles the `setxattr` system call (allows setting a limited subset of restricted extended attributes)
-security.syscalls.whitelist                     | string    | -                 | no            | A '\n' separated list of syscalls to whitelist (mutually exclusive with security.syscalls.blacklist\*)
-snapshots.schedule                              | string    | -                 | no            | Cron expression (`<minute> <hour> <dom> <month> <dow>`)
-snapshots.schedule.stopped                      | bool      | false             | no            | Controls whether or not stopped instances are to be snapshoted automatically
-snapshots.pattern                               | string    | snap%d            | no            | Pongo2 template string which represents the snapshot name (used for scheduled snapshots and unnamed snapshots)
-snapshots.expiry                                | string    | -                 | no            | Controls when snapshots are to be deleted (expects expression like `1M 2H 3d 4w 5m 6y`)
-user.\*                                         | string    | -                 | n/a           | Free form user key/value storage (can be used in search)
+Key                                         | Type      | Default           | Live update   | Condition     | Description
+:--                                         | :---      | :------           | :----------   | :----------       | :----------
+boot.autostart                              | boolean   | -                 | n/a           | -                 | Always start the instance when LXD starts (if not set, restore last state)
+boot.autostart.delay                        | integer   | 0                 | n/a           | -                 | Number of seconds to wait after the instance started before starting the next one
+boot.autostart.priority                     | integer   | 0                 | n/a           | -                 | What order to start the instances in (starting with highest)
+boot.host\_shutdown\_timeout                | integer   | 30                | yes           | -                 | Seconds to wait for instance to shutdown before it is force stopped
+boot.stop.priority                          | integer   | 0                 | n/a           | -                 | What order to shutdown the instances (starting with highest)
+environment.\*                              | string    | -                 | yes (exec)    | -                 | key/value environment variables to export to the instance and set on exec
+limits.cpu                                  | string    | - (all)           | yes           | -                 | Number or range of CPUs to expose to the instance
+limits.cpu.allowance                        | string    | 100%              | yes           | -                 | How much of the CPU can be used. Can be a percentage (e.g. 50%) for a soft limit or hard a chunk of time (25ms/100ms)
+limits.cpu.priority                         | integer   | 10 (maximum)      | yes           | -                 | CPU scheduling priority compared to other instances sharing the same CPUs (overcommit) (integer between 0 and 10)
+limits.disk.priority                        | integer   | 5 (medium)        | yes           | -                 | When under load, how much priority to give to the instance's I/O requests (integer between 0 and 10)
+limits.kernel.\*                            | string    | -                 | no            | container         | This limits kernel resources per instance (e.g. number of open files)
+limits.memory                               | string    | - (all)           | yes           | -                 | Percentage of the host's memory or fixed value in bytes (various suffixes supported, see below)
+limits.memory.enforce                       | string    | hard              | yes           | container         | If hard, instance can't exceed its memory limit. If soft, the instance can exceed its memory limit when extra host memory is available
+limits.memory.hugepages                     | boolean   | false             | no            | virtual-machine   | Controls whether to back the instance using hugepages rather than regular system memory
+limits.memory.swap                          | boolean   | true              | yes           | -                 | Whether to allow some of the instance's memory to be swapped out to disk
+limits.memory.swap.priority                 | integer   | 10 (maximum)      | yes           | -                 | The higher this is set, the least likely the instance is to be swapped to disk (integer between 0 and 10)
+limits.network.priority                     | integer   | 0 (minimum)       | yes           | -                 | When under load, how much priority to give to the instance's network requests (integer between 0 and 10)
+limits.processes                            | integer   | - (max)           | yes           | container         | Maximum number of processes that can run in the instance
+linux.kernel\_modules                       | string    | -                 | yes           | container         | Comma separated list of kernel modules to load before starting the instance
+migration.incremental.memory                | boolean   | false             | yes           | container         | Incremental memory transfer of the instance's memory to reduce downtime
+migration.incremental.memory.goal           | integer   | 70                | yes           | container         | Percentage of memory to have in sync before stopping the instance
+migration.incremental.memory.iterations     | integer   | 10                | yes           | container         | Maximum number of transfer operations to go through before stopping the instance
+nvidia.driver.capabilities                  | string    | compute,utility   | no            | container         | What driver capabilities the instance needs (sets libnvidia-container NVIDIA\_DRIVER\_CAPABILITIES)
+nvidia.runtime                              | boolean   | false             | no            | container         | Pass the host NVIDIA and CUDA runtime libraries into the instance
+nvidia.require.cuda                         | string    | -                 | no            | container         | Version expression for the required CUDA version (sets libnvidia-container NVIDIA\_REQUIRE\_CUDA)
+nvidia.require.driver                       | string    | -                 | no            | container         | Version expression for the required driver version (sets libnvidia-container NVIDIA\_REQUIRE\_DRIVER)
+raw.apparmor                                | blob      | -                 | yes           | container         | Apparmor profile entries to be appended to the generated profile
+raw.idmap                                   | blob      | -                 | no            | container         | Raw idmap configuration (e.g. "both 1000 1000")
+raw.lxc                                     | blob      | -                 | no            | container         | Raw LXC configuration to be appended to the generated one
+raw.qemu                                    | blob      | -                 | no            | virtual-machine   | Raw Qemu configuration to be appended to the generated command line
+raw.seccomp                                 | blob      | -                 | no            | container         | Raw Seccomp configuration
+security.devlxd                             | boolean   | true              | no            | -                 | Controls the presence of /dev/lxd in the instance
+security.devlxd.images                      | boolean   | false             | no            | -                 | Controls the availability of the /1.0/images API over devlxd
+security.idmap.base                         | integer   | -                 | no            | container         | The base host ID to use for the allocation (overrides auto-detection)
+security.idmap.isolated                     | boolean   | false             | no            | container         | Use an idmap for this instance that is unique among instances with isolated set
+security.idmap.size                         | integer   | -                 | no            | container         | The size of the idmap to use
+security.nesting                            | boolean   | false             | yes           | -                 | Support running lxd (nested) inside the instance
+security.privileged                         | boolean   | false             | no            | container         | Runs the instance in privileged mode
+security.protection.delete                  | boolean   | false             | yes           | -                 | Prevents the instance from being deleted
+security.protection.shift                   | boolean   | false             | yes           | container         | Prevents the instance's filesystem from being uid/gid shifted on startup
+security.secureboot                         | boolean   | true              | no            | virtual-machine   | Controls whether UEFI secure boot is enabled with the default Microsoft keys
+security.syscalls.blacklist                 | string    | -                 | no            | container         | A '\n' separated list of syscalls to blacklist
+security.syscalls.blacklist\_compat         | boolean   | false             | no            | container         | On x86\_64 this enables blocking of compat\_\* syscalls, it is a no-op on other arches
+security.syscalls.blacklist\_default        | boolean   | true              | no            | container         | Enables the default syscall blacklist
+security.syscalls.intercept.mknod           | boolean   | false             | no            | container         | Handles the `mknod` and `mknodat` system calls (allows creation of a limited subset of char/block devices)
+security.syscalls.intercept.mount           | boolean   | false             | no            | container         | Handles the `mount` system call
+security.syscalls.intercept.mount.allowed   | string    | -                 | yes           | container         | Specify a comma-separated list of filesystems that are safe to mount for processes inside the instance
+security.syscalls.intercept.mount.fuse      | string    | -                 | yes           | container         | Whether to mount shiftfs on top of filesystems handled through mount syscall interception
+security.syscalls.intercept.mount.shift     | boolean   | false             | yes           | container         | Whether to redirect mounts of a given filesystem to their fuse implemenation (e.g. ext4=fuse2fs)
+security.syscalls.intercept.setxattr        | boolean   | false             | no            | container         | Handles the `setxattr` system call (allows setting a limited subset of restricted extended attributes)
+security.syscalls.whitelist                 | string    | -                 | no            | container         | A '\n' separated list of syscalls to whitelist (mutually exclusive with security.syscalls.blacklist\*)
+snapshots.schedule                          | string    | -                 | no            | -                 | Cron expression (`<minute> <hour> <dom> <month> <dow>`)
+snapshots.schedule.stopped                  | bool      | false             | no            | -                 | Controls whether or not stopped instances are to be snapshoted automatically
+snapshots.pattern                           | string    | snap%d            | no            | -                 | Pongo2 template string which represents the snapshot name (used for scheduled snapshots and unnamed snapshots)
+snapshots.expiry                            | string    | -                 | no            | -                 | Controls when snapshots are to be deleted (expects expression like `1M 2H 3d 4w 5m 6y`)
+user.\*                                     | string    | -                 | n/a           | -                 | Free form user key/value storage (can be used in search)
 
 The following volatile keys are currently internally used by LXD:
 
@@ -221,17 +221,17 @@ lxc profile device add <profile> <name> <type> [key=value]...
 ## Device types
 LXD supports the following device types:
 
-ID (database)   | Name                              | Description
-:--             | :--                               | :--
-0               | [none](#type-none)                | Inheritance blocker
-1               | [nic](#type-nic)                  | Network interface
-2               | [disk](#type-disk)                | Mountpoint inside the instance
-3               | [unix-char](#type-unix-char)      | Unix character device
-4               | [unix-block](#type-unix-block)    | Unix block device
-5               | [usb](#type-usb)                  | USB device
-6               | [gpu](#type-gpu)                  | GPU device
-7               | [infiniband](#type-infiniband)    | Infiniband device
-8               | [proxy](#type-proxy)              | Proxy device
+ID (database)   | Name                              | Condition     | Description
+:--             | :--                               | :--           | :--
+0               | [none](#type-none)                | -             | Inheritance blocker
+1               | [nic](#type-nic)                  | -             | Network interface
+2               | [disk](#type-disk)                | -             | Mountpoint inside the instance
+3               | [unix-char](#type-unix-char)      | container     | Unix character device
+4               | [unix-block](#type-unix-block)    | container     | Unix block device
+5               | [usb](#type-usb)                  | container     | USB device
+6               | [gpu](#type-gpu)                  | container     | GPU device
+7               | [infiniband](#type-infiniband)    | container     | Infiniband device
+8               | [proxy](#type-proxy)              | container     | Proxy device
 
 ### Type: none
 A none type device doesn't have any property and doesn't create anything inside the instance.
@@ -252,12 +252,13 @@ LXD supports different kind of network devices:
  - [sriov](#nictype-sriov): Passes a virtual function of an SR-IOV enabled physical network device into the instance.
  - [routed](#nictype-routed): Creates a virtual device pair to connect the host to the instance and sets up static routes and proxy ARP/NDP entries to allow the instance to join the network of a designated parent interface.
 
+Currently, only the `bridged` type is supported with virtual machines.
+
 Different network interface types have different additional properties.
 
 Each possible `nictype` value is documented below along with the relevant properties for nics of that type.
 
 #### nictype: physical
-
 Straight physical device passthrough from the host. The targeted device will vanish from the host and appear in the instance.
 
 Device configuration properties:
@@ -273,7 +274,6 @@ maas.subnet.ipv4        | string    | -                 | no        | MAAS IPv4
 maas.subnet.ipv6        | string    | -                 | no        | MAAS IPv6 subnet to register the instance in
 
 #### nictype: bridged
-
 Uses an existing bridge on the host and creates a virtual device pair to connect the host bridge to the instance.
 
 Device configuration properties:
@@ -299,7 +299,6 @@ maas.subnet.ipv4         | string    | -                 | no        | MAAS IPv4
 maas.subnet.ipv6         | string    | -                 | no        | MAAS IPv6 subnet to register the instance in
 
 #### nictype: macvlan
-
 Sets up a new network device based on an existing one but using a different MAC address.
 
 Device configuration properties:
@@ -315,7 +314,6 @@ maas.subnet.ipv4        | string    | -                 | no        | MAAS IPv4
 maas.subnet.ipv6        | string    | -                 | no        | MAAS IPv6 subnet to register the instance in
 
 #### nictype: ipvlan
-
 Sets up a new network device based on an existing one using the same MAC address but a different IP.
 
 LXD currently supports IPVLAN in L3S mode.
@@ -352,7 +350,6 @@ ipv6.address            | string    | -                 | no        | Comma deli
 vlan                    | integer   | -                 | no        | The VLAN ID to attach to
 
 #### nictype: p2p
-
 Creates a virtual device pair, putting one side in the instance and leaving the other side on the host.
 
 Device configuration properties:
@@ -370,7 +367,6 @@ ipv4.routes             | string    | -                 | no        | Comma deli
 ipv6.routes             | string    | -                 | no        | Comma delimited list of IPv6 static routes to add on host to nic
 
 #### nictype: sriov
-
 Passes a virtual function of an SR-IOV enabled physical network device into the instance.
 
 Device configuration properties:
@@ -387,7 +383,6 @@ maas.subnet.ipv4        | string    | -                 | no        | MAAS IPv4
 maas.subnet.ipv6        | string    | -                 | no        | MAAS IPv6 subnet to register the instance in
 
 #### nictype: routed
-
 This NIC type is similar in operation to IPVLAN, in that it allows an instance to join an external network without needing to configure a bridge and shares the host's MAC address.
 
 However it differs from IPVLAN because it does not need IPVLAN support in the kernel and the host and instance can communicate with each other.
@@ -553,6 +548,9 @@ Example command.
 lxc config device add <instance> config disk source=cloud-init:config
 ```
 
+Currently only the root disk (path=/) and config drive (source=cloud-init:config) are supported with virtual machines.
+
+
 The following properties exist:
 
 Key                 | Type      | Default   | Required  | Description


More information about the lxc-devel mailing list