<div dir="ltr"><div>Aw, shucks, it was looking like it might work - but no joy. </div><div><br></div><div>root@olympia:~# lxc config set core.https_address <a href="http://192.168.111.193:8443">192.168.111.193:8443</a></div><div>root@olympia:~# lxc move kangal lxd1:</div><div>error: Error transferring container data: migration restore failed</div><div>(00.192913) 1: Error (mount.c:2406): mnt: Can't mount at ./sys/kernel/debug: Invalid argument</div><div>(00.206829) Error (cr-restore.c:1352): 23251 killed by signal 9</div><div>(00.255466) Error (cr-restore.c:2182): Restoring FAILED.</div><div><br></div><div>Any ideas from the error message? The only thing I see in the logs is this, only sending side:<br></div><div><div>Aug 17 14:49:15 olympia kernel: [317636.764104] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317636.897001] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317636.965993] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.025985] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.099008] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.223028] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.276026] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.407065] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.466059] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.559652] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.671028] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:16 olympia kernel: [317637.760296] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:17 olympia kernel: [317637.840036] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:17 olympia kernel: [317637.944061] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:17 olympia kernel: [317638.017056] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:17 olympia kernel: [317638.074090] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:20 olympia kernel: [317640.800234] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:20 olympia kernel: [317641.335276] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:25 olympia kernel: [317646.079503] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:25 olympia kernel: [317646.273487] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:25 olympia kernel: [317646.444487] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:25 olympia kernel: [317646.694511] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:26 olympia kernel: [317647.107553] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:27 olympia kernel: [317648.322648] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:27 olympia kernel: [317648.549680] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:27 olympia kernel: [317648.606610] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:28 olympia kernel: [317649.377678] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:28 olympia kernel: [317649.445660] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:28 olympia kernel: [317649.509650] cgroup: new mount options do not match the existing superblock, will be ignored</div><div>Aug 17 14:49:29 olympia kernel: [317649.921865] Netfilter messages via NETLINK v0.30.</div><div>Aug 17 14:49:29 olympia kernel: [317649.946436] ctnetlink v0.93: registering with nfnetlink.</div><div>Aug 17 14:49:29 olympia kernel: [317650.033557] br0: port 3(vethVHQ0XE) entered disabled state</div><div>Aug 17 14:49:29 olympia kernel: [317650.039297] device vethVHQ0XE left promiscuous mode</div><div>Aug 17 14:49:29 olympia kernel: [317650.039302] br0: port 3(vethVHQ0XE) entered disabled state</div><div>Aug 17 14:49:30 olympia kernel: [317651.765330] audit_printk_skb: 69 callbacks suppressed</div><div>Aug 17 14:49:30 olympia kernel: [317651.765333] audit: type=1400 audit(1471470570.966:77): apparmor="STATUS" operation="profile_remove" info="profile does not exist" error=-2 profile="unconfined" name="lxd-kangal_</var/lib/lxd>" pid=25080 comm="apparmor_parser"</div></div><div><br></div><div><br></div><div>And this, on the receiving side:</div><div><div>Aug 17 14:49:56 ronnie kernel: [107460.037655] audit: type=1400 audit(1471470596.676:89): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-kangal_</var/lib/lxd>" pid=23239 comm="apparmor_parser"</div><div>Aug 17 14:49:56 ronnie NetworkManager[2696]: <warn> [1471470596.8817] device (vethIA06PF): failed to find device 11 'vethIA06PF' with udev</div><div>Aug 17 14:49:56 ronnie NetworkManager[2696]: <info> [1471470596.8828] manager: (vethIA06PF): new Veth device (/org/freedesktop/NetworkManager/Devices/10)</div><div>Aug 17 14:49:56 ronnie NetworkManager[2696]: <info> [1471470596.8991] devices added (path: /sys/devices/virtual/net/vethIA06PF, iface: vethIA06PF)</div><div>Aug 17 14:49:56 ronnie NetworkManager[2696]: <info> [1471470596.8991] device added (path: /sys/devices/virtual/net/vethIA06PF, iface: vethIA06PF): no ifupdown configuration found.</div><div>Aug 17 14:49:57 ronnie NetworkManager[2696]: <info> [1471470597.0303] device (vethIA06PF): driver 'veth' does not support carrier detection.</div><div>Aug 17 14:49:57 ronnie NetworkManager[2696]: <info> [1471470597.0351] devices removed (path: /sys/devices/virtual/net/vethIA06PF, iface: vethIA06PF)</div></div><div><br></div><div><br></div><div><br></div><div>Jake</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 17, 2016 at 2:50 PM, jjs - mainphrame <span dir="ltr"><<a href="mailto:jjs@mainphrame.com" target="_blank">jjs@mainphrame.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks Stephane, that goes to the root of the problem. I must have been thinking in the back of my mind that I'd already set that, but come to think of it, that was back when this box was running 14.04. before a clean install of 16.04<div><br></div><div>Jake</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 17, 2016 at 1:45 PM, Stéphane Graber <span dir="ltr"><<a href="mailto:stgraber@ubuntu.com" target="_blank">stgraber@ubuntu.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">So one way around the problem would be with:<br>
<br>
lxc config set core.https_address <a href="http://192.168.111.193:8443" rel="noreferrer" target="_blank">192.168.111.193:8443</a><br>
<br>
on your olympia host. This will force LXD to use that IP during<br>
container transfer and should fix your problem.<br>
<div><div><br>
On Wed, Aug 17, 2016 at 01:28:52PM -0700, jjs - mainphrame wrote:<br>
> Hi Stephane -<br>
><br>
> lxd1, lxd2, and kangal are all on the same lan, and connectivity is good:<br>
><br>
> root@olympia:~# ssh kangal w<br>
> root@kangal's password:<br>
> 13:26:53 up 3 days, 14:50, 0 users, load average: 0.29, 0.37, 0.37<br>
> USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT<br>
><br>
> root@ronnie:~# ssh kangal w<br>
> root@kangal's password:<br>
> 13:27:09 up 3 days, 14:51, 0 users, load average: 0.29, 0.36, 0.37<br>
> USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT<br>
><br>
> Regards,<br>
><br>
> Jake<br>
><br>
><br>
> On Wed, Aug 17, 2016 at 1:24 PM, Stéphane Graber <<a href="mailto:stgraber@ubuntu.com" target="_blank">stgraber@ubuntu.com</a>><br>
> wrote:<br>
><br>
> > On Wed, Aug 17, 2016 at 01:14:43PM -0700, jjs - mainphrame wrote:<br>
> > > Greetings,<br>
> > ><br>
> > > I'm running lxd version 2.0.3-0ubuntu1~ubuntu16.04.2<br>
> > ><br>
> > > I'm trying to get lxd to correctly execute a move of a container from one<br>
> > > lxd host to another. I have two ubuntu 16.04 hosts, ronnie (designated as<br>
> > > lxd1) and olympia (designated as lxd2):<br>
> > ><br>
> > > root@olympia:~# lxc remote list<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | NAME | URL | PROTOCOL<br>
> > > | PUBLIC | STATIC |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | images | <a href="https://images.linuxcontainers.org" rel="noreferrer" target="_blank">https://images.linuxcontainers<wbr>.org</a> | lxd<br>
> > > | YES | NO |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | local (default) | unix:// | lxd<br>
> > > | NO | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | lxd1 | <a href="https://192.168.111.20:8443" rel="noreferrer" target="_blank">https://192.168.111.20:8443</a> | lxd<br>
> > > | NO | NO |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | ubuntu | <a href="https://cloud-images.ubuntu.com/releases" rel="noreferrer" target="_blank">https://cloud-images.ubuntu.co<wbr>m/releases</a> |<br>
> > > simplestreams | YES | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | ubuntu-daily | <a href="https://cloud-images.ubuntu.com/daily" rel="noreferrer" target="_blank">https://cloud-images.ubuntu.co<wbr>m/daily</a> |<br>
> > > simplestreams | YES | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > ><br>
> > > root@ronnie:~# lxc remote list<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | NAME | URL | PROTOCOL<br>
> > > | PUBLIC | STATIC |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | images | <a href="https://images.linuxcontainers.org" rel="noreferrer" target="_blank">https://images.linuxcontainers<wbr>.org</a> | lxd<br>
> > > | YES | NO |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | local (default) | unix:// | lxd<br>
> > > | NO | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
</div></div><span>> > > | lxd2 | <a href="https://192.168.111.193:8443" rel="noreferrer" target="_blank">https://192.168.111.193:8443</a> | lxd<br>
> > > | NO | NO |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
</span><span>> > > | ubuntu | <a href="https://cloud-images.ubuntu.com/releases" rel="noreferrer" target="_blank">https://cloud-images.ubuntu.co<wbr>m/releases</a> |<br>
> > > simplestreams | YES | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > > | ubuntu-daily | <a href="https://cloud-images.ubuntu.com/daily" rel="noreferrer" target="_blank">https://cloud-images.ubuntu.co<wbr>m/daily</a> |<br>
> > > simplestreams | YES | YES |<br>
> > > +-----------------+-----------<wbr>------------------------------<br>
> > -+---------------+--------+---<wbr>-----+<br>
> > ><br>
</span><div><div>> > > You can see that the remotes are configured with their local lan<br>
> > addresses.<br>
> > > So far so good?<br>
> > ><br>
> > ><br>
> > ><br>
> > > Here are the 2 containers currently on lxd2:<br>
> > > root@olympia:~# lxc list<br>
> > > +--------+---------+----------<wbr>-------------+------+---------<br>
> > ---+-----------+<br>
> > > | NAME | STATE | IPV4 | IPV6 | TYPE |<br>
> > SNAPSHOTS |<br>
> > > +--------+---------+----------<wbr>-------------+------+---------<br>
> > ---+-----------+<br>
> > > | akita | RUNNING | 192.168.111.22 (eth0) | | PERSISTENT | 0<br>
> > |<br>
> > > +--------+---------+----------<wbr>-------------+------+---------<br>
> > ---+-----------+<br>
> > > | kangal | RUNNING | 192.168.111.44 (eth0) | | PERSISTENT | 0<br>
> > |<br>
> > > +--------+---------+----------<wbr>-------------+------+---------<br>
> > ---+-----------+<br>
> > ><br>
> > ><br>
> > ><br>
> > > Now, I try to move a container from lxd2 to lxd1:<br>
> > > root@olympia:~# lxc move kangal lxd1:<br>
> > > error: Error transferring container data: Unable to connect to:<br>
> > > <a href="http://192.168.1.8:8443" rel="noreferrer" target="_blank">192.168.1.8:8443</a><br>
> ><br>
> > Is there any way for lxd1 to connect to kangal?<br>
> ><br>
> > The way LXD currently deals with cross-host communication is that the<br>
> > client has the source host issue a token which the client sends to the<br>
> > target along with instructions on how to connect to the source.<br>
> ><br>
> > The target then directly connects to the source to fetch the data<br>
> > (in this case, the container).<br>
> ><br>
> > This means that there must be a way for the target to connect to the<br>
> > source on the LXD port without being blocked by firewalls or going<br>
> > through NAT.<br>
> ><br>
> ><br>
> > Some more details can be found at<br>
> > <a href="https://www.stgraber.org/2016/04/12/lxd-2-0-remote-hosts-" rel="noreferrer" target="_blank">https://www.stgraber.org/2016/<wbr>04/12/lxd-2-0-remote-hosts-</a><br>
> > and-container-migration-612/<br>
> > in the "network requirements" and "how this all works" sections.<br>
> ><br>
> ><br>
> > The client is currently supposed to iterate through all the IPs that the<br>
> > source server advertises (see addresses field in "lxc info kangal"), the<br>
> > one that's in the error message is usually the last one of those.<br>
> ><br>
> ><br>
> > If core.https_address is set on the source host, then only that address<br>
> > will be attempted since it's the only one LXD will be listening on.<br>
> ><br>
> ><br>
> > As mentioned in the blog post, we do have a plan to improve the<br>
> > situation by having the client relay the data in cases where the two<br>
> > servers can't talk, but we haven't made much progress on implementing<br>
> > that so far.<br>
> ><br>
> > ><br>
> > > Why is it trying to connect to 192.168.1.8? That is a local wireless<br>
> > > address on lxd2, but it was never mentioned in any lxd configuration:<br>
> > ><br>
> > > root@olympia:~# ifconfig wlp3s0<br>
> > > wlp3s0 Link encap:Ethernet HWaddr 80:56:f2:05:ce:6c<br>
> > > inet addr:192.168.1.8 Bcast:192.168.1.255 Mask:255.255.255.0<br>
> > > inet6 addr: fe80::8256:f2ff:fe05:ce6c/64 Scope:Link<br>
> > > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br>
> > > RX packets:401712 errors:0 dropped:0 overruns:0 frame:0<br>
> > > TX packets:219214 errors:0 dropped:0 overruns:0 carrier:0<br>
> > > collisions:0 txqueuelen:1000<br>
> > > RX bytes:95097222 (95.0 MB) TX bytes:34201360 (34.2 MB)<br>
> > ><br>
> > ><br>
> > > So my question is, how do we get lxd to ignore the local wireless IP, and<br>
> > > execute the lxc move command using the configured IPs?<br>
> > ><br>
> > > Regards,<br>
> > ><br>
> > > Jake<br>
> ><br>
> > > ______________________________<wbr>_________________<br>
> > > lxc-users mailing list<br>
> > > <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainer<wbr>s.org</a><br>
> > > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.o<wbr>rg/listinfo/lxc-users</a><br>
> ><br>
> ><br>
> > --<br>
> > Stéphane Graber<br>
> > Ubuntu developer<br>
> > <a href="http://www.ubuntu.com" rel="noreferrer" target="_blank">http://www.ubuntu.com</a><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > lxc-users mailing list<br>
> > <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainer<wbr>s.org</a><br>
> > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.o<wbr>rg/listinfo/lxc-users</a><br>
> ><br>
<br>
> ______________________________<wbr>_________________<br>
> lxc-users mailing list<br>
> <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainer<wbr>s.org</a><br>
> <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.o<wbr>rg/listinfo/lxc-users</a><br>
<br>
<br>
--<br>
Stéphane Graber<br>
Ubuntu developer<br>
<a href="http://www.ubuntu.com" rel="noreferrer" target="_blank">http://www.ubuntu.com</a><br>
</div></div><br>______________________________<wbr>_________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainer<wbr>s.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.o<wbr>rg/listinfo/lxc-users</a><br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>