[lxc-users] Migrating LXD VMs/containers between nodes

[Serge Hallyn]

Quoting Dilvan Moreira (dilvan at gmail.com):
> On Wed, Nov 25, 2015 at 3:50 PM, Serge Hallyn <serge.hallyn at ubuntu.com>
> wrote:
> 
> > Quoting Dilvan Moreira (dilvan at gmail.com):
> > >    1.
> > >    Is there a way to pass a block volume (Ceph/rbd) or folder to a LXD
> > VM,
> >
> > You can just have your scripts mount the volume/folder to
> > /var/lib/lxd/containers/container/rootfs.  You can set
> > lxc.rootfs= in the raw.lxc.
> 
> 
>    Where this raw.lxc file is located (Can I have one for each VM)? In any

Yes, you can have one per vm.

lxc config set c21 raw.lxc lxc.aa_profile=unconfined

> case, that doesn’t solve my problem. I cannot just start a VM from another
> machine (as I can do now), I will have to copy the metadata.yam file and
> templates folder. Don’t they change over time? Is it safe to copy them just
> after the container is created?

After creation the metadata.yaml and templates should be irrelevant.  If
you make changes to the container config, though, those would need to be
propagated to all the other lxd dbs.

> >   Or you can wait for
> >
> > https://github.com/lxc/lxc/pull/670
> >
> > Once that gets into lxc, I've not thought through whether you'll need
> > any specific support from lxd.
> >
> > I think what you're doing now makes a lot of sense - the only shortcoming
> > is that live migration may be complicated because of the shared rootfs.
> >
> > Note that Guido (cc:d) is also doing a similar setup (using NFS).
> >
> > >    so LXD itself would mount the root folder (/)? It would be nice to
> > have
> > >    a block volume for each VM (and not all VMs using the same folder).
> > >    2.
> > >
> > >    Is there a simpler solution than OpenStack for my use case (or a
> > simpler
> > >    installation procedure for OpenStack)?
> >
> > imho juju is the simplest way to install openstack.
> >
> 
>    In all examples I found of juju installing OpenStack (I don’t know much
> about juju), it was using a set of VMs or MaaS (Metal as a Service). VMs
> would be pointless in my case and MaaS is very complicated to implement
> (and not very useful in my case). Can I use juju in real bare metal? If so,
> can you point me to some example/documentation?

Hm, you can target juju at lxc (and experimentally at lxd), and I know some
parts of openstack can be placed in containers, but I don't know, if you are
setting up openstack with https://github.com/lxc/nova-lxd , whether all of
the openstack pieces can be in containers.  Chuck?

> > >    3.
> > >
> > >    Ultimately, I would like my cluster to have the ability to schedule
> > VMs
> > >    to nodes, move VMs from failed nodes, etc. Any suggestions on how to
> > get
> > >    that?
> >
> > The biggest complication I see is that when you ask lxd to move a
> > container,
> > lxd will want to migrate the rootfs.  I'm waiting for clarification from
> > Tycho on whether lxd will be smart enough to DTRT.
> >
> 
>    Using Ceph/rbd, there is no need to migrate anything.

Of course, that was my point :)  The smarts to not migrate anything are not
in lxd yet.

> I am not aiming to
> have live migration, so the plan is to shutdown the container and start it
> agin in another node.

ok.