[lxc-users] Containers have network issues when their host uses a bonded interface

[Bostjan Skufca]

Hi Peter,

since you mentioned you are using bridged interfaces, is my assumption
correct that your containers's network connection is joined directly
to this bridge and containers talk to the world direcly (L2) and not
via routed (L3) network over host OS?

Did you try using routed setup (using bond0 directly and creating
dedicated br1just  for containers) and taking bridging functionality
of the linux kernel out of the picture?

I am very interested in your findings, as I have similar setup planned
for deployment in the coming weeks (vlan trunk -> bond -> bridge ->
containers).

(I had similar issues on bonded interfaces on VMware, where tap-based
OpenVPN server would not work at all. It had to do something with how
vmware handles bonds and this not being compatible with L2 stuff
coming out of VM. The second thing I remember very vaguely is that I
tried using bridged interface inside container too, and it did not
cooperate well with host's bridge, so I stopped using bridges inside
containers altogether.)

b.


On 8 September 2015 at 14:46, Peter Steele <pwsteele at gmail.com> wrote:
> I don't think I'm up to the challenge of an ASCII art representation of our
> cluster. How's this instead?
>
> We run on a cluster of 1U and 2U servers. Each server is connected to the
> same network by either multiple 1GigE or 10GigE links and is configured with
> a bonded/bridged interface. Every server runs some number of containers,
> based on resources available and the configuration parameters we select when
> installing the cluster. Almost all network traffic is between the various
> containers, both within the same host as well as between containers running
> on different hosts. There is minimal communication in our software between
> containers and the servers themselves. We have some protocols based on UDP
> and a few based on TCP. Pretty standard stuff ultimately.
>
> We run this identical configuration with VMs (KVM) without issues. We're
> moving to containers since our benchmarks indicate we'd gain some in
> performance.
>
> On 09/07/2015 10:35 PM, Guido Jäkel wrote:
>>
>> Dear Peter,
>>
>> may you paint a small ASCII art of your desired network setup and the
>> network configuration? I see no reason that "LXC", i.e. the veth connecting
>> two differently namespaced IP stacks is responsible for any problems.
>>
>> BTW: I'm using a complex network setup with no problems, too. A real NIC
>> is connected to a selected trunk of VLANs; they are decoded and connected to
>> a couple of bridges and the containers are attached to one or more of it as
>> needed for the business case.
>>
>> greetings
>>
>> Guido
>>
>>
>> On 07.09.2015 20:49, Peter Steele wrote:
>>>
>>> We're having issues with networking connections in our containers when
>>> the host is configured with bonded interfaces. When we configure these same
>>> servers to run with VMs, everything works fine, but when we swap out the VMs
>>> for equivalently configured containers, we get all kinds of network
>>> connection failures between containers. The network issues go away when we
>>> reduce the bond to have a single member. More specifically, we configure our
>>> server with a host bridge (br0), where the host bridge links to the host's
>>> bonded interface (bond0). We've been using this model for years with VMs.
>>> Something isn't working quite right with containers though.
>>>
>>> Is there something we need to set in our containers' config files when
>>> the host is setup with bonding?
>>>
>>> _______________________________________________
>>> lxc-users mailing list
>>> lxc-users at lists.linuxcontainers.org
>>> http://lists.linuxcontainers.org/listinfo/lxc-users
>>
>> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users