[lxc-users] Trying out migration, getting "error: checkpoint failed"

[Giles Thomas]

Hi Tycho,

On 05/05/15 14:43, Tycho Andersen wrote:
> On Tue, May 05, 2015 at 02:18:23PM +0100, Giles Thomas wrote:
>> On 04/05/15 14:15, Tycho Andersen wrote:
>>> What versions of criu and liblxc do you have installed? Can you look in
>>> /var/lib/lxd/lxc/<container>/lxc.log (or /var/log/lxd/<container>/lxc.log;
>>> there was a bug fixed recently that moved it to the latter location).
>> Heh, I hadn't realised that criu wasn't automatically installed as a
>> dependency for lxd (might be worth adding to the blog post?), so I didn't
>> have it installed at all.
> Ah yeah. It seems I forgot to mention anything about criu at all :)
>
>> However, installing it doesn't fix the problem.  I currently have criu
>> 1.4-1~ubuntu14.04.1~ppa1 and liblxc1
>> 1.1.2+master~20150428-0938-0ubuntu1~trusty.   The former was installed just
>> with an apt-get install criu, and the latter via following your blog post
>> pretty much robotically...
>>
>> /var/log/lxd/migratee/lxc.log is present, but empty.
> Does /var/lib/lxd/lxc/migratee/lxc.log exist?

Aha!  It doesn't, but there is a file /var/lib/lxd/lxc/migratee/log, 
which I missed last time around.   I just tried the move again, with a 
tail -f on that file running in the background, and got some output:

     root at XXXXXXX:/var/log/lxd/migratee# lxc move lxd:migratee lxd2:migratee
                 lxc 1430839338.008 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.010 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.010 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
                 lxc 1430839338.010 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.012 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.014 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.014 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
                 lxc 1430839338.016 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.019 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.020 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.020 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
                 lxc 1430839338.022 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.025 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.027 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.027 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
                 lxc 1430839338.028 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.887 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.889 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839338.889 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
                 lxc 1430839340.029 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839340.031 DEBUG    lxc_commands - 
commands.c:lxc_cmd_handler:888 - peer has disconnected
                 lxc 1430839340.031 DEBUG    lxc_commands - 
commands.c:lxc_cmd_get_state:574 - 'migratee' is in 'RUNNING' state
     error: checkpoint failed
     root at XXXXXXX:/var/log/lxd/migratee#

Anything useful there?  Or anything that points at any other logs I 
should be checking?

> You might try building criu from their git, I'm not sure the version
> in our PPA is new enough to actually work with liblxc 1.1.2.

Sure, I can definitely give that a go -- I'll wait for half an hour or 
so, though, just in case you see the log above and immediately realise 
what stupid thing I've done wrong ;-)


All the best,

Giles

-- 
Giles Thomas <giles at pythonanywhere.com>

PythonAnywhere: Develop and host Python from your browser
<https://www.pythonanywhere.com/>

A product from PythonAnywhere LLP
17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number OC378414.
Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK