[lxc-users] Migrating LXD VMs/containers between nodes

[Serge Hallyn]

Quoting Dilvan Moreira (dilvan at gmail.com):
> Hi,
> 
> I am setting up a small cluster using a VM (master) and 3 bare metal
> servers (all running Ubuntu 14.04). Each bare metal server also has 2T of
> disk space exported using Ceph 0.94.5
> <http://docs.ceph.com/docs/v0.94.5/start/>.
> 
> I would like to be able to run LXD VMs (containers) on this cluster and to
> easily migrate these VMs to different nodes. I could have installed
> OpenStack, but that seems rather complicated to me (maybe OpenStack is an
> overkill for such a small cluster like mine). So, my solution was to create
> a big Ceph/rbd block volume and mount it at the LXD container folder (
> /var/lib/lxd/containers) in all nodes. To move a VM, I just shut it down in
> one node and then start it again in another.
> 
> For just one VM it's working fine, but it doesn't seem to me as a long term
> solution. My questions:
> 
>    1.
> 
>    Is there a way to pass a block volume (Ceph/rbd) or folder to a LXD VM,

You can just have your scripts mount the volume/folder to
/var/lib/lxd/containers/container/rootfs.  You can set
lxc.rootfs= in the raw.lxc.  Or you can wait for

https://github.com/lxc/lxc/pull/670

Once that gets into lxc, I've not thought through whether you'll need
any specific support from lxd.

I think what you're doing now makes a lot of sense - the only shortcoming
is that live migration may be complicated because of the shared rootfs.

Note that Guido (cc:d) is also doing a similar setup (using NFS).

>    so LXD itself would mount the root folder (/)? It would be nice to have
>    a block volume for each VM (and not all VMs using the same folder).
>    2.
> 
>    Is there a simpler solution than OpenStack for my use case (or a simpler
>    installation procedure for OpenStack)?

imho juju is the simplest way to install openstack.

>    3.
> 
>    Ultimately, I would like my cluster to have the ability to schedule VMs
>    to nodes, move VMs from failed nodes, etc. Any suggestions on how to get
>    that?

The biggest complication I see is that when you ask lxd to move a container,
lxd will want to migrate the rootfs.  I'm waiting for clarification from
Tycho on whether lxd will be smart enough to DTRT.