[lxc-users] Network instability with bridged nat and macvlan interfaces

[Andrey Repin]

Greetings, Michel Jansens!

> I’m running on Ubuntu18.04 LXC 3.0.0.

> I’ve created 5 debian9 containers with default eth0 networking on NAT:

> # lxc network show lxdbr0
> config:
>   ipv4.address: 10.1.1.1/24
>   ipv4.dhcp.ranges: 10.1.1.2-10.1.1.99
>   ipv4.nat: "true"
>   ipv6.address: fd42:6f79:c120:7701::1/64
>   ipv6.nat: "true"
> description: Natted network 0
> name: lxdbr0
> type: bridge

> One of the containers (frontal) has an additional interface configured with:

> # lxc network attach vlan7 frontal
> # lxc config show kspreprodfrontal
> …
> devices:
>   vlan7:
>     nictype: macvlan
>     parent: vlan7
>     type: nic

> vlan7 is a flan with id: 7 configured in /etc/netplan/01-netcfg.yaml 
> ... 
> vlans:
>     vlan7:
>       id: 7
>       link: enp1s0f0

I'm no expert, frankly, but it itching me to mix brctl and macvlan like that.

> I’ve changed the frontal host internal networking so that eth1 comes first
> and default route is going through eth1. 
> Everything works internal and external…except from time to time, the
> frontal starts refusing connexions from the outside for a few seconds (up to 50).
> It looks like general networking because all ports suddenly stop working (connexion refused)
> internally the frontal remains reachable
> I’m running haproxy on ports 80 and 443, but also tried running apache2 on
> port 8082. All ports go down at the same time.

> I’ve now installed an Ubuntu (16.04) container and added the vlan7 network
> the same way.
> It worked fine…for about an hour and stopped working again, but for good.
> What is weird is that port 80 and 443 are refused but port 22 is working
> (maybe that’s the host ssh?).

> Any idea?

Your explanation is not very clear in parts where you describe the failure.

> Thanks for any suggestion.

My first suggestion would be to rebuild your networking a little bit
different.

1. Create a dummy internal interface and bind your containers' macvlan bridges
  to it. Bind an additional bridged macvlan on host to be able to reach into
  the containers' network.
2. If your vlan7 is a dedicate network interface for your containers, pass it
  as physical to the ingress container.


-- 
With best regards,
Andrey Repin
Thursday, June 7, 2018 18:26:48

Sorry for my terrible english...