[lxc-users] live migration using lxd 0.4

[Tycho Andersen]

Hi KATOH,

On Thu, Mar 26, 2015 at 04:52:17PM +0900, KATOH Yasufumi wrote:
> >>> On Tue, 24 Mar 2015 08:34:36 -0600
>     in message   "Re: [lxc-users] live migration using lxd 0.4"
>                   Tycho Andersen-san wrote:
> 
> > Sorry about that. This was a bug that I inadvertently introduce just
> > before 0.4 was released. It is fixed by
> > 141734d852b9ffb34eca10366e96d57c93ff23e8, which went in shortly after
> > 0.4 was tagged, and will be in 0.5 due to be released today.
> 
> Thank you for your reply! I'm trying to do live migration using 0.5
> now :-) But I have not succeeded yet. (^^;)

I'm working on getting everything working again with privileged LXC
(several recent changes have broken things), once that happens the LXD
bits are in place to make it will be easy to do migration in LXD.

> | 2015/03/26 16:41:59 operation %!s(func()
> |  shared.OperationResult=0x4c5d20) finished: { checkpoint failed}
> 
> > You'll also find that you need some other stuff to do live migration
> > (not documented anywhere, unfortunately):
> 
> > 1. you can't use lxcfs (or any other fuse fs)
> 
> I stopped lxcfs.
> 
> > 2. only non-uidmapped containers work (you can do this in lxd by just
> >    setting the uidmap for lxd to be 0-65536, or by setting the uidmap
> >    in the container's raw.lxc config)
> 
> I have a question. What does non-uidmapped container means? Should I
> configure "security.privileged true"? Would you please tell me the way
> to configure it specifically?

I just tried to do this, and it appears to be broken. I sent a PR to
fix it,

https://github.com/lxc/lxd/pull/438

So you'll need that as well as an LXC built with my two recent c/r
patches to the lxc-devel list (which haven't been merged yet). If you
have all that, I think you can do something like

lxc profile create migratable
lxc profile edit migratable

# paste the content below; these are a bunch of fixes for current criu
# limitations.

name: migratable
config:
  raw.lxc: |
    lxc.console = none
    lxc.cgroup.devices.deny = c 5:1 rwm
    lxc.start.auto =
    lxc.start.auto = proc:mixed sys:mixed
  security.privileged: "true"
devices: {}

Then do:

# assuming lxd2 is your target remote
lxc profile copy migratable lxd2:

lxc init ubuntu migratee
lxc config profile apply migratee migratable

lxc start migratee

# assuming lxd2 is the http:// url for your default lxd
lxc move lxd:migratee lxd2:migratee

And you should see it try to migrate. Currently it will still fail because of
external bind mounts like binfmt and some others. I'm working on a patch for
this now, but you could disable the additional bind mounts if you're not using
them and it should work.

Tycho

> > 3. you can't be using systemd on the host
> 
> I use busybox and ubuntu container. :)
> 
> > 4. probably some others that I've forgotten about at the moment :)
> 
> > Thanks for trying the code though! I'll hopefully be sending out fixes
> > for some of this stuff in the coming weeks. Let me know if you have
> > any problems and I can try and give you workarounds.
> 
> > Tycho
> 
> Thanks,
> KATOH Yasufumi
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users