[lxc-users] lxc progress and a few questions

Tue Mar 29 16:48:05 UTC 2016

On Tue, Mar 29, 2016 at 09:30:19AM -0700, jjs - mainphrame wrote:
> On Tue, Mar 29, 2016 at 7:18 AM, Tycho Andersen <
> tycho.andersen at canonical.com> wrote:
> 
> > On Mon, Mar 28, 2016 at 08:47:24PM -0700, jjs - mainphrame wrote:
> > > I've looked at ct migration between 2 ubuntu 16.04 hosts today, and had
> > > some interesting problems;  I find that migration of stopped containers
> > > works fairly reliably; but live migration, well, it transfers a lot of
> > > data, then exits with a failure message. I can then move the same
> > > container, stopped, with no problem.
> > >
> > > The error is the same every time, a failure of "mkdtemp" -
> >
> > It looks like your host /tmp isn't writable by the uid map that the
> > container is being restored as?
> >
> 
> Which is odd, since /tmp has 1777 perms on both hosts, so I don't see how
> it could be a permissions problem. Surely the default apparmor profile is
> not the cause? You did give me a new idea though, and I'll set up a test
> with privileged containers for comparison. Is there a switch to enable
> verbose logging?

It already is enabled, you can find the full logs in
/var/log/lxd/$container/migration_*

Perhaps the pwd of the CRIU task is what's broken instead, since CRIU
isn't supplying a full mkdtemp template. I'll have a deeper look in a
bit.

Tycho

> 
> > >
> > > root at ronnie:~# lxc move third lxd:
> > > error: Error transferring container data: restore failed:
> > > (00.033172)      1: Error (cr-restore.c:1489): mkdtemp failed
> > > crtools-proc.x9p5OH: Permission denied
> > > (00.060072) Error (cr-restore.c:1352): 9188 killed by signal 9
> > > (00.117126) Error (cr-restore.c:2182): Restoring FAILED.
> > >
> > > root at raskolnikov:~# lxc move third lxd2:
> > > error: Error transferring container data: restore failed:
> > > (00.039099)      1: Error (cr-restore.c:1489): mkdtemp failed
> > > crtools-proc.a3U2t5: Permission denied
> > > (00.063015) Error (cr-restore.c:1352): 1771 killed by signal 9
> > > (00.115261) Error (cr-restore.c:2182): Restoring FAILED.
> > >
> > > root at ronnie:~# lxc move third lxd:
> > > error: Error transferring container data: restore failed:
> > > (00.034542)      1: Error (cr-restore.c:1489): mkdtemp failed
> > > crtools-proc.gee5YS: Permission denied
> > > (00.059955) Error (cr-restore.c:1352): 9274 killed by signal 9
> > > (00.109272) Error (cr-restore.c:2182): Restoring FAILED.
> > >
> > >
> > > Thanks for any pointers you can provide -
> > >
> > > PS  - on a side note, I'd originally set up the new box with a separate
> > > /var partition on btrfs. As a result, "criu check" would die instantly
> > with
> > > a segmentation error. After putting /var back on / (ext4) criu check
> > > produces the message "Looks good."
> >
> > Hmm. Can you get the backtrace?
> >
> 
> Tycho
> >
> 
> Sorry to say, I moved on quickly and the btrfs filesystem is no more, but I
> can recreate that scenario again later.
> 
> 
> Regards,
> 
> Jake
> 
> 
> 
> >
> > > Jake
> > >
> > >
> > >
> > > On Mon, Mar 28, 2016 at 7:47 AM, Tycho Andersen <
> > > tycho.andersen at canonical.com> wrote:
> > >
> > > > On Sun, Mar 27, 2016 at 09:22:44PM -0700, jjs - mainphrame wrote:
> > > > > You've had some success with live migrations? At any rate, I'm
> > upgrading
> > > > my
> > > > > ubuntu 15.10 test box to 16.04 so that I'll have 2 of them. I'll
> > give it
> > > > a
> > > > > whirl.
> > > >
> > > > Yes, there are still various unsupported kernel features, but it
> > > > should work out of the box for a modern stock xenial image on a xenial
> > > > host. There is one more xenial kernel patch which will eventually be
> > > > released that will break things, but I'm in the process of upstreaming
> > > > some CRIU patches to handle that case, and we'll distro patch those
> > > > when they're ready.
> > > >
> > > > Tycho
> > > >
> > > > > On Sun, Mar 27, 2016 at 9:20 PM, Fajar A. Nugraha <list at fajar.net>
> > > > wrote:
> > > > >
> > > > > > On Sun, Mar 27, 2016 at 11:31 PM, jjs - mainphrame <
> > jjs at mainphrame.com
> > > > >
> > > > > > wrote:
> > > > > > > The 2nd link you sent seems to indicate that
> > > > > > > live migration wants to work, but I haven't been able to find any
> > > > reports
> > > > > > > from normal users in the field who've actually succeeded with
> > live
> > > > > > > migration. if I've missed something, please let me know.
> > > > > >
> > > > > > My best advice is to try yourself with latest 16.04 daily.
> > > > > >
> > > > > > I've had both success and failure with it.
> > > > > >
> > > > > > --
> > > > > > Fajar
> > > > > > _______________________________________________
> > > > > > lxc-users mailing list
> > > > > > lxc-users at lists.linuxcontainers.org
> > > > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> > > > > >
> > > >
> > > > > _______________________________________________
> > > > > lxc-users mailing list
> > > > > lxc-users at lists.linuxcontainers.org
> > > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> > > >
> > > > _______________________________________________
> > > > lxc-users mailing list
> > > > lxc-users at lists.linuxcontainers.org
> > > > http://lists.linuxcontainers.org/listinfo/lxc-users
> > > >
> >
> > > _______________________________________________
> > > lxc-users mailing list
> > > lxc-users at lists.linuxcontainers.org
> > > http://lists.linuxcontainers.org/listinfo/lxc-users
> >
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >

> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users