[lxc-users] lxc progress and a few questions

Wed Apr 6 22:18:54 UTC 2016

Greetings -

I'be not yet been able to reproduce that one shining moment from Mar
29 when live migration of privileged containers was working, under
kernel 4.4.0-15

To recap. live container migration broke with 4.4.0-16, and is still
broken in 4.4.0-17 - but  now, instead of producing an error message,
an attempt to live migrate a container merely hangs forever. Is that
expected, or should I be seeing something more? BTW - the migration
dump log for that container hasn't been touched for a week. I'll be
glad to supply more info if this is not a known issue.

Recent updates seem to have created a new problem. the CTs which
configure their own network settings work (aside from migration) but
none of the CTs which depend on dhcp are getting IPs. BTW I'm using a
bridge connected to my local network and dhcp, not the default lxc
dhcp server. I see the packets on the host bridge, but they don't
reach the dhcp server. I'd be curious to know if there have been any
dhcp issues since recent updates. If not, I'll need to troubleshoot
other causes, but it's odd that dhcp simply stops working for all CTs
on both lxd hosts after updates.

Jake

On Wed, Mar 30, 2016 at 6:27 AM, Tycho Andersen
<tycho.andersen at canonical.com> wrote:
> On Tue, Mar 29, 2016 at 11:17:26PM -0700, jjs - mainphrame wrote:
>> Well, I've found some interesting things here today. I created a couple of
>> privileged xenial containers, and sure enough, I was able to live migrate
>> them back and forth between the 2 lxd hosts.
>>
>> So far, so good.
>>
>> Then I did an apt upgrade - among the changes was a kernel change from
>>  4.4.0-15 to 4.4.0-16 - and live migration stopped working.
>>
>> Here are the failure messages that resulted from attempting the very same
>> live migrations that worked before the upgrade and reboot into 4.4.0-16:
>>
>> root at raskolnikov:~# lxc move akira lxd2:
>> error: Error transferring container data: checkpoint failed:
>> (00.092234) Error (mount.c:740): mnt: 83:./sys/fs/cgroup/devices doesn't
>> have a proper root mount
>> (00.098187) Error (cr-dump.c:1600): Dumping FAILED.
>>
>>
>> root at ronnie:~# lxc move third lxd:
>> error: Error transferring container data: checkpoint failed:
>> (00.076107) Error (mount.c:740): mnt: 326:./sys/fs/cgroup/perf_event
>> doesn't have a proper root mount
>> (00.080388) Error (cr-dump.c:1600): Dumping FAILED.
>
> Yep, this is a known issue with -16. We need both a kernel patch and a
> patch to CRIU before it will start working again. I have a branch at:
>
> https://github.com/tych0/criu/tree/cgroup-root-mount
>
> which should work if you want to keep playing with it, but hopefully
> we'll have the situation sorted out in the next few days.
>
> Tycho
>
>> Jake
>>
>> PS - Thanks for the html mail heads-up - I've been using google mail
>> services for this domain. I'll have to look into the config options, and
>> see if I can do the needful.
>
>>
>> On Tue, Mar 29, 2016 at 12:45 PM, Andrey Repin <anrdaemon at yandex.ru> wrote:
>>
>> > Greetings, jjs - mainphrame!
>> >
>> > >> On Mon, Mar 28, 2016 at 08:47:24PM -0700, jjs - mainphrame wrote:
>> >  >>> I've looked at ct migration between 2 ubuntu 16.04 hosts today, and
>> > had
>> >  >>> some interesting problems;  I find that migration of stopped
>> > containers
>> >  >>> works fairly reliably; but live migration, well, it transfers a lot of
>> >  >>> data, then exits with a failure message. I can then move the same
>> >  >>> container, stopped, with no problem.
>> >  >>>
>> >  >>> The error is the same every time, a failure of "mkdtemp" -
>> > >>
>> > >>  It looks like your host /tmp isn't writable by the uid map that the
>> > >>  container is being restored as?
>> >
>> >
>> > > Which is odd, since /tmp has 1777 perms on both hosts, so I don't see how
>> > > it could be a permissions problem. Surely the default apparmor profile is
>> > > not the cause? You did give me a new idea though, and I'll set up a test
>> > > with privileged containers for comparison. Is there a switch to enable
>> > verbose logging?
>> >
>> > I've ran into the same issue once. Stumbled upon it for nearly a month,
>> > falsely
>> > blaming LXC.
>> > Recreating a container's rootfs from scratch resolved the issue.
>> > I know not of what caused it to begin with, must've been some kind of
>> > glitch.
>> >
>> > P.S.
>> > It would be great if you can configure your mail client to not use HTML
>> > format
>> > for lists.
>> >
>> >
>> > --
>> > With best regards,
>> > Andrey Repin
>> > Tuesday, March 29, 2016 22:43:04
>> >
>> > Sorry for my terrible english...
>> > _______________________________________________
>> > lxc-users mailing list
>> > lxc-users at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-users
>> >
>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users