[lxc-users] LXC networking stop working between containers and real network

[Ruzsinszky Attila]

Hi,

There is an Ubuntu 14.04 64 bit up to date host.
LXC version is: 2.0.3 (from backport packages)
OpenvSwitch: 2.0.2.

Container1: Ubuntu 14.04
Container2: Ubuntu 16.04 (both of them was installed from root.fs.zx,
because lxc-create doesn't work with auth. Squid proxy)

Both containers are working perfectly in "standalone" mode.
I use lxcbr0 as a bridge between the containers. There is dnsmasq for DHCP
and it is working, because containers get IP address (from 10.0.3.0/24
range).
There is an OVS bridge: vbr0 and its port is lxcbr0 on the host. The real
Ethernet interface is: eth0 which is connected to the real network. There
is a mgmtlxc0 virt. management interface which IP is: 10.0.3.2/24. I can
ping every machine in the 10.0.3.0/24 range.
The MAC addresses of the containers are different. I checked them.
mgmtlxc0 and the lxcbr0 are tagged for VLAN (tag=800 in OVS config)

I want to MASQUERADE the lxc-net to the real network:
Chain POSTROUTING (policy ACCEPT 54626 packets, 5252K bytes)
 pkts bytes target     prot opt in     out     source
destination
  246 20520 MASQUERADE  all  --  *      *       10.0.3.0/24         !
10.0.3.0/24

Routing table:
root at fcubi:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
default        real_router     0.0.0.0         UG    0      0        0 eth0
LXCnet          *               255.255.255.0   U     0      0        0
mgmtlxc0
FCnet           *               255.255.255.0   U     1      0        0 eth0

The problem is:
I try to ping from container1 (lub4) to a host on the real network. It is
working.
I try to ping from container2 (lub5) to the same host and it is not
working! The DNS resolving is OK, but no answer from the real host.

I checked the traffic on eth0 on lub4 or 5 (inside the containers). I can
see the ICMP echo REQ packets.
They are arrived to the host's lxcbr0 interface. I think it is good.
I checked the hosts's mgmtlxc0 interface which is the routing interface on
IP level. I can see the REQ packets.
ip4_forwarding is enabled (=1).
The next interface is eth0 and no traffic from containers on it! I filtered
for ICMP and no REQ! So the host "filter out" (or not routing) my MASQUed
ICMP packets.
I think it is not a MASQ problem, because without MASQUERADING I had to see
the outgoing REQ packets with wrong source IP (10.0.3.x) and of course
there won't be any answer because the real host knows nothing about routing
to 10.0.3.0 lxcnet. But no any outgoing packets.
I tried to remove the all iptables rules except MASQ and nothing was
changed.

If I ping between lub4 and 5 it is working (virtual) when the real not.

If I restart the containers one by one and I change the ping test (1st is
lub5 and the 2nd is lub4) the 2nd won't ping so not depend ont the
containers OS version.

I think the problem maybe in MASQ or routing between mgmtlxc0 and eth0.
netstat-nat doesn't work and I don't know why.
Do you have any clue?

I've got another host which is Fedora 23 64 bit (OVS 2.5) with 3 U14.04
containers and it seems working.

I'll do some more test. For example making a new U14.04 container because
on F23 the container's versions are the same.
LXD was installed but not used or configured.

TIA,
Ruzsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160719/a96e9d73/attachment.html>