[lxc-users] Trying out migration, getting "error: checkpoint failed"

[Tycho Andersen]

On Wed, May 06, 2015 at 08:40:52AM -0600, Tycho Andersen wrote:
> Hi Giles,
> 
> On Tue, May 05, 2015 at 05:10:56PM +0100, Giles Thomas wrote:
> > Hi Tycho,
> > 
> > On 05/05/15 16:50, Tycho Andersen wrote:
> > >Can you check the lxd stderr by chance (probably lives in /var/log
> > >somewhere depending on what init system you're using)? I suspect that
> > >liblxc is rejecting dumping the container in its internal predump checks,
> > >but the above log doesn't say way unfortunately. Sorry for all the
> > >confusion, the logging stuff here is still a bit of a mess, although a bit
> > >better on the current lxd master.
> > 
> > Oddly, there didn't appear to be one; "find /var/log -name \*lxd\*" just
> > found "/var/log/lxd".  Nothing relevant-looking in "/var/log/upstart/" apart
> > from "lxc-net.log", which has an "Address already in use" error:
> > 
> >     dnsmasq: failed to create listening socket for 10.0.3.1: Address already
> > in use
> >     Failed to setup lxc-net.
> > 
> > Doubly-oddly, there's a "/etc/init/lxd.conf" *and* a "/etc/init.d/lxd",
> > which confuses me a little.  Does that not mean that both init and upstart
> > will try to start it?  (My knowledge of the workings of init systems in not
> > as in-depth as I would like.) Should I remove one of them then change the
> > remaining one to write stdout/err somewhere sensible?
> 
> You could, but it may be easier to just stop the lxd service and run
> it manually so that it writes stderr to the terminal you're using.
> 
> Looking at the code path, it looks like there are a few (really
> unlikely) ways it could fail without writing anything to the log (such
> as OOM or not being able to make a temporary directory, but it's root
> so as long as you have enough disk/ram it /should/ die with some error
> message). If you can't find anything, it may be worth building a
> liblxc from source and trying to debug things that way.

Sorry, I did just find one notable exception with the current git
master: liblxc doesn't complain when excing criu fails. Do you have
criu installed in a place where liblxc can find it?

I posted a patch to fix this particular case, but it seems likely
that's where your problem is.

Tycho

> > I can also see that there are still init and upstart scripts for lxcfs,
> > which is a bit messy -- the "apt-get remove lxcfs" should presumably have
> > deleted them -- but they depend on "/usr/bin/lxcfs", which definitely
> > doesn't exist, so I guess that's not the problem.
> 
> `remove` doesn't always remove config files, `purge` is supposed to
> though.
> 
> Tycho
> 
> > 
> > All the best,
> > 
> > Giles
> > 
> > -- 
> > Giles Thomas <giles at pythonanywhere.com>
> > 
> > PythonAnywhere: Develop and host Python from your browser
> > <https://www.pythonanywhere.com/>
> > 
> > A product from PythonAnywhere LLP
> > 17a Clerkenwell Road, London EC1M 5RD, UK
> > VAT No.: GB 893 5643 79
> > Registered in England and Wales as company number OC378414.
> > Registered address: 28 Ely Place, 3rd Floor, London EC1N 6TD, UK
> > 
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users