<div dir="ltr"><div class="gmail_default" style="font-size:small">The 6.7 solution from Vmware is very expensive in terms of licensing, nobody is going to upgrade because of that. The 6.5 solution is useless because the mac addresses never age, they accumulate. The real solution is to use a kernel 5.X plus ipvlan, and I already know it works fine, no need to set the whole network in promiscuous mode. It should be ideal if LXC would make ipvlan work for real, meanwhile, just add many ipvlans interfaces to the host, and export each one to a different container as type=phys. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 24, 2020 at 8:20 AM Michael Honeyman <<a href="mailto:michael@honeyman.net.au">michael@honeyman.net.au</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I don't often write to this list so apologies as I'm probably messing up the thread somehow.</div><div><br></div><div>Saint Michael wrote: "...
Vmware only allows multiple macs if the entire network is set in promiscuous mode..."</div><div><br></div><div>Not strictly LXC/LXD related, but VMware has implemented two solutions to this problem since 6.5. They first released the MAC-learning dVfilter fling which still requires promiscuous mode but removed the flooding behaviour (more like a filtered hub than a switch - not sure if this improves the performance problem).</div><div><br></div><div>There is also the Learnswitch which requires a distributed virtual switch, but implements proper MAC flooding and learning, which removes the requirement for promiscuous mode. This allows the VM to have multiple MACs behind one NIC, just as you'd expect on a physical network. This fling was released as a standard feature in 6.7, but as it requires DVS it is unfortunately locked behind a license. I haven't seen if the MAC-learning dVfilter fling has been ported to vSphere 6.7 yet or not.</div><div><br></div><div>Hope that helps,</div><div>Michael.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 24 Mar 2020 at 23:00, <<a href="mailto:lxc-users-request@lists.linuxcontainers.org" target="_blank">lxc-users-request@lists.linuxcontainers.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send lxc-users mailing list submissions to<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:lxc-users-request@lists.linuxcontainers.org" target="_blank">lxc-users-request@lists.linuxcontainers.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:lxc-users-owner@lists.linuxcontainers.org" target="_blank">lxc-users-owner@lists.linuxcontainers.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of lxc-users digest..."<br>
Today's Topics:<br>
<br>
1. Re: Networking (Fajar A. Nugraha)<br>
2. Re: Networking (Saint Michael)<br>
3. Re: Networking (Serge E. Hallyn)<br>
4. Re: Networking (Saint Michael)<br>
5. Re: Networking (Fajar A. Nugraha)<br>
6. Re: Networking (Saint Michael)<br>
<br><br><br>---------- Forwarded message ----------<br>From: "Fajar A. Nugraha" <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Mon, 23 Mar 2020 19:26:18 +0700<br>Subject: Re: [lxc-users] Networking<br>On Fri, Mar 20, 2020 at 5:36 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
><br>
> I use plain LXC, not LXD. is ipvlan supported?<br>
<br>
<a href="https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html" rel="noreferrer" target="_blank">https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html</a><br>
<br>
-- <br>
Fajar<br>
<br>
<br><br><br>---------- Forwarded message ----------<br>From: Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Mon, 23 Mar 2020 09:15:57 -0400<br>Subject: Re: [lxc-users] Networking<br><div dir="ltr"><div style="font-size:small">As I said, type=ipvlan does not work on the latest version if LXC from git. BUT there is a workaround: create as many ipvlan interfaces as you need at the host level, which shall be used later as type="phys" networking on containers. That works.</div><div style="font-size:small"><br></div><div style="font-size:small"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 23, 2020 at 8:26 AM Fajar A. Nugraha <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, Mar 20, 2020 at 5:36 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
><br>
> I use plain LXC, not LXD. is ipvlan supported?<br>
<br>
<a href="https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html" rel="noreferrer" target="_blank">https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html</a><br>
<br>
-- <br>
Fajar<br>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
</blockquote></div>
<br><br><br>---------- Forwarded message ----------<br>From: "Serge E. Hallyn" <<a href="mailto:serge@hallyn.com" target="_blank">serge@hallyn.com</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Mon, 23 Mar 2020 11:37:18 -0500<br>Subject: Re: [lxc-users] Networking<br>Hi,<br>
<br>
just to make sure i understand right - you mean it is not supported in<br>
lxc-user-nic? And never was, so not a regression?<br>
<br>
Or has something regressed?<br>
<br>
On Mon, Mar 23, 2020 at 09:15:57AM -0400, Saint Michael wrote:<br>
> As I said, type=ipvlan does not work on the latest version if LXC from git.<br>
> BUT there is a workaround: create as many ipvlan interfaces as you need at<br>
> the host level, which shall be used later as type="phys" networking on<br>
> containers. That works.<br>
> <br>
> <br>
> <br>
> On Mon, Mar 23, 2020 at 8:26 AM Fajar A. Nugraha <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>> wrote:<br>
> <br>
> > On Fri, Mar 20, 2020 at 5:36 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
> > ><br>
> > > I use plain LXC, not LXD. is ipvlan supported?<br>
> ><br>
> > <a href="https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html" rel="noreferrer" target="_blank">https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html</a><br>
> ><br>
> > --<br>
> > Fajar<br>
> > _______________________________________________<br>
> > lxc-users mailing list<br>
> > <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
> > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
> ><br>
<br>
> _______________________________________________<br>
> lxc-users mailing list<br>
> <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
> <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
<br>
<br>
<br><br><br>---------- Forwarded message ----------<br>From: Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Mon, 23 Mar 2020 12:48:34 -0400<br>Subject: Re: [lxc-users] Networking<br><div dir="ltr"><div style="font-size:small">It is supported, there is no error, but there is no communication at all with the gateway. If you start the same exact network configuration in the container with the type=phys, it works fine, ergo, the issue is type=ipvlan.</div><div style="font-size:small"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Mar 23, 2020 at 12:37 PM Serge E. Hallyn <<a href="mailto:serge@hallyn.com" target="_blank">serge@hallyn.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
just to make sure i understand right - you mean it is not supported in<br>
lxc-user-nic? And never was, so not a regression?<br>
<br>
Or has something regressed?<br>
<br>
On Mon, Mar 23, 2020 at 09:15:57AM -0400, Saint Michael wrote:<br>
> As I said, type=ipvlan does not work on the latest version if LXC from git.<br>
> BUT there is a workaround: create as many ipvlan interfaces as you need at<br>
> the host level, which shall be used later as type="phys" networking on<br>
> containers. That works.<br>
> <br>
> <br>
> <br>
> On Mon, Mar 23, 2020 at 8:26 AM Fajar A. Nugraha <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>> wrote:<br>
> <br>
> > On Fri, Mar 20, 2020 at 5:36 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
> > ><br>
> > > I use plain LXC, not LXD. is ipvlan supported?<br>
> ><br>
> > <a href="https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html" rel="noreferrer" target="_blank">https://linuxcontainers.org/lxc/manpages//man5/lxc.container.conf.5.html</a><br>
> ><br>
> > --<br>
> > Fajar<br>
> > _______________________________________________<br>
> > lxc-users mailing list<br>
> > <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
> > <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
> ><br>
<br>
> _______________________________________________<br>
> lxc-users mailing list<br>
> <a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
> <a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
<br>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
</blockquote></div>
<br><br><br>---------- Forwarded message ----------<br>From: "Fajar A. Nugraha" <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Tue, 24 Mar 2020 15:19:57 +0700<br>Subject: Re: [lxc-users] Networking<br>On Mon, Mar 23, 2020 at 11:48 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
><br>
> It is supported, there is no error, but there is no communication at all with the gateway. If you start the same exact network configuration in the container with the type=phys, it works fine, ergo, the issue is type=ipvlan.<br>
<br>
"exact network configuration" inside the container? I'm pretty sure it<br>
would fail.<br>
<br>
If you read what I wrote earlier:<br>
"<br>
set /etc/resolv.conf on the container manually, and disable network<br>
interface setup inside the container.<br>
"<br>
<br>
This works in my test (using lxc 3.2.1 from<br>
<a href="https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/daily" rel="noreferrer" target="_blank">https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/daily</a>):<br>
# Network configuration<br>
<a href="http://lxc.net.0.name" rel="noreferrer" target="_blank">lxc.net.0.name</a> = eth0<br>
lxc.net.0.type = ipvlan<br>
lxc.net.0.ipvlan.mode = l3s<br>
lxc.net.0.l2proxy = 1<br>
lxc.net.0.link = eth0<br>
lxc.net.0.ipv4.gateway = dev<br>
lxc.net.0.ipv4.address = <a href="http://10.0.3.222/32" rel="noreferrer" target="_blank">10.0.3.222/32</a><br>
lxc.net.0.flags = up<br>
<br>
<br>
While inside the container, setup resolv.conf manually, and disable<br>
networking setup (e.g. removing everything under /etc/netplan/ on<br>
ubuntu should work).<br>
<br>
Common issue with macvlan/ipvlan of "container not being able to<br>
contact the host" would still apply.<br>
<br>
-- <br>
Fajar<br>
<br>
<br><br><br>---------- Forwarded message ----------<br>From: Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>><br>To: LXC users mailing-list <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br>Cc: <br>Bcc: <br>Date: Tue, 24 Mar 2020 07:22:02 -0400<br>Subject: Re: [lxc-users] Networking<br><div dir="ltr"><div style="font-size:small">That scheme in my case would not work. I have two interfaces inside the container, and each one talks to a different network, for business reasons. I use policy-based-routing to make sure that packets go to the right places. I need that the container can hold a full configuration. In my case, I use ifupdown, not netplan, since my containers are for an older version of Debian.</div><div style="font-size:small">It is "not right" that ipvlan does not work out-of-the-box like macvlan or veth. Somebody has to fix it. I cannot use macvlan because Vmware only allows multiple macs if the entire network is set in promiscuous mode, and that kills performance. So basically the only workaround is ipvlan. As I said, if you use type=phys and ipvlan inside the host, it works fine, without altering the container.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Mar 24, 2020 at 4:20 AM Fajar A. Nugraha <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Mon, Mar 23, 2020 at 11:48 PM Saint Michael <<a href="mailto:venefax@gmail.com" target="_blank">venefax@gmail.com</a>> wrote:<br>
><br>
> It is supported, there is no error, but there is no communication at all with the gateway. If you start the same exact network configuration in the container with the type=phys, it works fine, ergo, the issue is type=ipvlan.<br>
<br>
"exact network configuration" inside the container? I'm pretty sure it<br>
would fail.<br>
<br>
If you read what I wrote earlier:<br>
"<br>
set /etc/resolv.conf on the container manually, and disable network<br>
interface setup inside the container.<br>
"<br>
<br>
This works in my test (using lxc 3.2.1 from<br>
<a href="https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/daily" rel="noreferrer" target="_blank">https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/daily</a>):<br>
# Network configuration<br>
<a href="http://lxc.net.0.name" rel="noreferrer" target="_blank">lxc.net.0.name</a> = eth0<br>
lxc.net.0.type = ipvlan<br>
lxc.net.0.ipvlan.mode = l3s<br>
lxc.net.0.l2proxy = 1<br>
lxc.net.0.link = eth0<br>
lxc.net.0.ipv4.gateway = dev<br>
lxc.net.0.ipv4.address = <a href="http://10.0.3.222/32" rel="noreferrer" target="_blank">10.0.3.222/32</a><br>
lxc.net.0.flags = up<br>
<br>
<br>
While inside the container, setup resolv.conf manually, and disable<br>
networking setup (e.g. removing everything under /etc/netplan/ on<br>
ubuntu should work).<br>
<br>
Common issue with macvlan/ipvlan of "container not being able to<br>
contact the host" would still apply.<br>
<br>
-- <br>
Fajar<br>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
</blockquote></div>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
</blockquote></div></div>
_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br>
</blockquote></div>