[lxc-devel] [lxd/master] storage: allow attaching storage volumes

brauner on Github lxc-bot at linuxcontainers.org
Mon Jun 26 23:16:49 UTC 2017


A non-text attachment was scrubbed...
Name: not available
Type: text/x-mailbox
Size: 2636 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20170626/123f5d2b/attachment.bin>
-------------- next part --------------
From 0e07a4cda9a8e8cd28c2ca417d9fe31ecb0c1084 Mon Sep 17 00:00:00 2001
From: Christian Brauner <christian.brauner at ubuntu.com>
Date: Mon, 26 Jun 2017 21:53:22 +0200
Subject: [PATCH] storage: allow attaching storage volumes

Since it's implementation the LXD storage API has supported attaching storage
volumes to containers. One of the oustanding challenges before a working
implementation of the shiftfs concept in the kernel is how to deal with id
mappings when mounting custom storage volumes. This spec tries to propose a
viable solution that aims at security.

When a new custom storage volume is created:

    lxc storage volume create <pool> <volume>

and attached to a container for the first time:

    lxc storage volume attach <pool> <volume> <container> <path-to-attach-to>

it will not have any prior id mapping associated with it.

1. Attaching a previously not id-mapped storage volume
The first container to attach the storage volume will perform an id mapping on
the storage volume. The serialized id mapping recorded in the containers config
file will then be stored as a volatile key in the storage volume's config.

2. Attaching an already id-mapped storage volume
When a storage volume is attached to a container it is checked whether the
volatile key is empty. If it is empty case 1. applies. If it is not empty it is
checked whether the id mapping of the container matched the idmap recorded in
the volatile key. If they match the container is allowed to attach the storage
volume. If they do not match it will be checked whether the storage volume is
atttached to any other containers. If it is attached to any other containers the
container is not allowed to attach the storage volume and an appriorate error is
returned. If it is not attached to any other containers the container is allowed
to attach the storage volume, map the ids and update the volatile key
appropriately.

3. Attaching a storage volume to multiple containers
When a storage volume is attached to multiple containers all containers must
have the same id mapping. This consensus must be upheld as long as multiple
containers share the same storage volume.

3. Changing the idmapping of a container with attached id-mapped storage volumes
If the idmapping of a container is changed that has an id-mapped storage volume
attached to it which is shared among multiple containers the change will not be
applied an approriate error is returned.

Closes #3389.

Signed-off-by: Christian Brauner <christian.brauner at ubuntu.com>
---
 lxd/container_lxc.go          |  2 +-
 lxd/storage.go                | 83 +++++++++++++++++++++++++++++++++++++++++++
 lxd/storage_volumes_config.go |  1 +
 3 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/lxd/container_lxc.go b/lxd/container_lxc.go
index d0bd3648e..e8a5c03a1 100644
--- a/lxd/container_lxc.go
+++ b/lxd/container_lxc.go
@@ -6156,7 +6156,7 @@ func (c *containerLXC) createDiskDevice(name string, m types.Device) (string, er
 		// Initialize a new storage interface and check if the
 		// pool/volume is mounted. If it is not, mount it.
 		volumeType, _ := storagePoolVolumeTypeNameToType(volumeTypeName)
-		s, err := storagePoolVolumeInit(c.daemon, m["pool"], volumeName, volumeType)
+		s, err := storagePoolVolumeAttachInit(c.daemon, m["pool"], volumeName, volumeType, c.idmapset)
 		if err != nil && !isOptional {
 			return "", fmt.Errorf("Failed to initialize storage volume \"%s\" of type \"%s\" on storage pool \"%s\": %s.",
 				volumeName,
diff --git a/lxd/storage.go b/lxd/storage.go
index 085872643..6d28084e3 100644
--- a/lxd/storage.go
+++ b/lxd/storage.go
@@ -387,6 +387,89 @@ func storagePoolInit(d *Daemon, poolName string) (storage, error) {
 	return storageInit(d, poolName, "", -1)
 }
 
+func storagePoolVolumeAttachInit(d *Daemon, poolName string, volumeName string, volumeType int, idmapset *shared.IdmapSet) (storage, error) {
+	st, err := storageInit(d, poolName, volumeName, volumeType)
+	if err != nil {
+		return nil, err
+	}
+
+	var lastIdmap *shared.IdmapSet
+	poolVolumePut := st.GetStoragePoolVolumeWritable()
+	if poolVolumePut.Config["volatile.idmap"] != "" {
+		// Convert the volume type name to our internal integer representation.
+		volumeTypeName, err := storagePoolVolumeTypeToName(volumeType)
+		if err != nil {
+			return nil, err
+		}
+
+		volumeUsedBy, err := storagePoolVolumeUsedByGet(d, volumeName, volumeTypeName)
+		if err != nil {
+			return nil, err
+		}
+
+		lastIdmap = new(shared.IdmapSet)
+		err = json.Unmarshal([]byte(poolVolumePut.Config["volatile.idmap"]), &lastIdmap)
+		if err != nil {
+			return nil, err
+		}
+
+		if !reflect.DeepEqual(idmapset, lastIdmap) {
+			if len(volumeUsedBy) > 0 {
+				return nil, fmt.Errorf("idmaps are not identical")
+			}
+		} else {
+			return st, nil
+		}
+	}
+
+	idmapBytes, err := json.Marshal(idmapset)
+	if err != nil {
+		return nil, err
+	}
+	jsonIdmap := string(idmapBytes)
+	poolVolumePut.Config["volatile.idmap"] = jsonIdmap
+
+	st.SetStoragePoolVolumeWritable(&poolVolumePut)
+
+	err = storagePoolVolumeUpdate(d, poolName, volumeName, volumeType, poolVolumePut.Description, poolVolumePut.Config)
+	if err != nil {
+		return nil, err
+	}
+
+	ourMount, err := st.StoragePoolVolumeMount()
+	if err != nil {
+		return nil, err
+	}
+	if ourMount {
+		defer func () {
+			_, err := st.StoragePoolVolumeUmount()
+			if err != nil {
+				logger.Warnf("failed to unmount storage volume")
+			}
+		}()
+	}
+
+	remapPath := getStoragePoolVolumeMountPoint(poolName, volumeName)
+
+	if lastIdmap != nil {
+		err := lastIdmap.UnshiftRootfs(remapPath)
+		if err != nil {
+			return nil, err
+		}
+		logger.Debugf("shifted storage volume")
+	}
+
+	if idmapset != nil {
+		err := idmapset.ShiftRootfs(remapPath)
+		if err != nil {
+			return nil, err
+		}
+		logger.Debugf("shifted storage volume")
+	}
+
+	return st, nil
+}
+
 func storagePoolVolumeInit(d *Daemon, poolName string, volumeName string, volumeType int) (storage, error) {
 	// No need to detect storage here, its a new container.
 	return storageInit(d, poolName, volumeName, volumeType)
diff --git a/lxd/storage_volumes_config.go b/lxd/storage_volumes_config.go
index 7fab8a743..96b8a0bf5 100644
--- a/lxd/storage_volumes_config.go
+++ b/lxd/storage_volumes_config.go
@@ -23,6 +23,7 @@ var storageVolumeConfigKeys = map[string]func(value string) error{
 	},
 	"zfs.use_refquota":     shared.IsBool,
 	"zfs.remove_snapshots": shared.IsBool,
+	"volatile.idmap":       shared.IsAny,
 }
 
 func storageVolumeValidateConfig(name string, config map[string]string, parentPool *api.StoragePool) error {


More information about the lxc-devel mailing list