<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Thanks Stéphane. Really appreciate the fast reply. Will be looking forward to the next code drop.<div class=""><br class=""><div class=""><br class=""></div><div class="">As for the macvlan issue, turns out the interface was plumbed but never turned up. After running “ifconfig vxlan1500 up”, I could get both containers pinging properly across the network.</div><div class=""><br class=""></div><div class="">For anyone else who might want to try VXLAN multicast between containers, here is a quick set of commands I used to get it working:</div><div class="">--------------------------------</div><div class="">ip route -4 add 239.0.0.1 eth1</div><div class="">ip link add vxlan1500 type vxlan group 239.0.0.1 dev eth1 dstport 0 id 1500</div><div class="">ifconfig vxlan1500 up</div><div class=""><edit LXD profile to match - set the nictype to “macvlan”, and the parent to “vxlan1500”> ></div><div class="">--------------------------------</div><div class=""><br class=""></div><div class="">Simply replace the “vxlan1500” with your interface name of choice and pick you physical ethernet port number (eth1 in the example above). The parameters “id 1500” specify the VXLAN Network ID (0-<span style="color: rgb(68, 68, 68); font-size: 13px; orphans: 2; white-space: pre-wrap; widows: 2;" class="">16777215</span>).</div><div class=""><br class=""></div><div class="">For what its worth, this is a huge win for me as I can setup a real environment using software-defined VLANs w/out modifying any top-of-rack switches. I simply create a new VXLAN segment for each new customer on our LXD servers and deploy a software firewall that manages traffic between the VXLAN segment with a local gateway.</div><div class=""><br class=""></div><div class="">Awesome!</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">-Ron</div><div class=""><br class=""></div><div class=""> </div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 23, 2017, at 5:00 PM, Stéphane Graber <<a href="mailto:stgraber@ubuntu.com" class="">stgraber@ubuntu.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi,<br class=""><br class="">I sent a pull request which allows oevrriding the interface in multicast mode:<br class=""> <a href="https://github.com/lxc/lxd/pull/3210" class="">https://github.com/lxc/lxd/pull/3210</a><br class=""><br class="">When writing that code, I did notice that in my earlier implementation I<br class="">always selected the default interface for those, so that explains why no<br class="">amount of routing trickery would help.<br class=""><br class="">Stéphane<br class=""><br class="">On Sun, Apr 23, 2017 at 04:36:43PM -0400, Ron Kelley wrote:<br class=""><blockquote type="cite" class="">Thanks for the speedy reply! From my testing, the VXLAN tunnel always seems to use eth0. After running the “ip -4 route add” command per your note below, I disabled eth1 on one of the hosts but was still able to ping between the two containers. I re-enabled that interface and disabled eth0; the ping stopped. It seems the VXLAN tunnel is bound to eth0.<br class=""><br class="">By chance, is there a workaround to make this work properly? I also tried using the macvlan interface type specifying a VXLAN tunnel interface and it would not work either. For clarity, this is what I did:<br class=""><br class="">ip link add vxlan500 type vxlan group 239.0.0.1 dev eth1 dstport 0 id 500<br class="">ip route -4 add 239.0.0.1 eth1<br class=""><edit the LXD default profile; set the nictype to “macvlan”, and the parent to “vxlan500”><br class=""><br class="">I was hoping a raw VXLAN interface would work instead of using the LXD create command.<br class=""><br class=""><br class="">-Ron<br class=""><br class=""><br class=""><blockquote type="cite" class="">On Apr 23, 2017, at 4:18 PM, Stéphane Graber <<a href="mailto:stgraber@ubuntu.com" class="">stgraber@ubuntu.com</a>> wrote:<br class=""><br class="">Hi,<br class=""><br class="">VXLAN in multicast mode (as is used in your case), when no multicast<br class="">address is specified will be using 239.0.0.1.<br class=""><br class="">This means that whatever route you have to reach "239.0.0.1" will be<br class="">used by the kernel for the VXLAN tunnel, or so would I expect.<br class=""><br class=""><br class="">Does:<br class="">ip -4 route add 239.0.0.1 dev eth1<br class=""><br class="">Cause the VXLAN traffic to now use eth1?<br class=""><br class="">If it doesn't, then that'd suggest that the multicast VXLAN interface<br class="">does in fact get tied to a particular parent interface and we should<br class="">therefore add an option to LXD to let you choose that interface.<br class=""><br class="">Stéphane<br class=""><br class="">On Sun, Apr 23, 2017 at 04:04:03PM -0400, Ron Kelley wrote:<br class=""><blockquote type="cite" class="">Greetings all.<br class=""><br class="">Following Stéphane’s excellent guide on using multicast VXLAN with LXD (<a href="https://stgraber.org/2016/10/27/network-management-with-lxd-2-3/" class="">https://stgraber.org/2016/10/27/network-management-with-lxd-2-3/</a>). In my lab, I have setup a few servers running Ubuntu 16.04 with LXD 2.12 and multiple interfaces (eth0, eth1, eth2). My goal is to setup a multi-tenant computing solution using VXLAN to separate network traffic. I want to dedicate eth0 as the mgmt-only interface and use eth1 (or other additional interfaces) as customer-only interfaces. I have read a number of guides but can’t find anything that clearly spells out how to create bridged interfaces using eth1, eth2, etc for LXD.<br class=""><br class="">I can get everything working using a single “eth0” interface on my LXD hosts using the following commands:<br class="">-----------------------------------------------------------<br class="">lxc network create vxlan100 ipv4.address=none ipv6.address=none tunnel.vxlan100.protocol=vxlan tunnel.vxlan100.id=100<br class="">lxc launch ubuntu: testvm01<br class="">lxc network attach vxlan100 testvm01<br class="">-----------------------------------------------------------<br class=""><br class="">All good so far. I created two test containers running on separate LXD servers using the above VXLAN ID and gave each a static IP Address (i.e.: 10.1.1.1/24 and 10.1.1.2/24). Both can ping back and forth. 100% working.<br class=""><br class="">The next step is to use eth1 instead of eth0 on my LXD servers, but I can’t find a keyword in the online docs that specify which interface to bind (<a href="https://github.com/lxc/lxd/blob/master/doc/networks.md" class="">https://github.com/lxc/lxd/blob/master/doc/networks.md</a>).<br class=""><br class="">Any pointers/clues?<br class=""><br class="">Thanks,<br class=""><br class="">-Ron<br class=""></blockquote></blockquote></blockquote>_______________________________________________<br class="">lxc-users mailing list<br class=""><a href="mailto:lxc-users@lists.linuxcontainers.org" class="">lxc-users@lists.linuxcontainers.org</a><br class="">http://lists.linuxcontainers.org/listinfo/lxc-users</div></div></blockquote></div><br class=""></div></div></body></html>