<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>
<div><div dir="ltr">Greetings,<div><br></div><div>I am trying to use Ubuntu Server 12.04 in a VMware vm as a test-bed for hosting several lxc containers while I work some configuration kinks out of them. Eventually I plan to host them directly on physical hardware.</div><div><br></div><div>Anyway, I am having trouble convincing the lxc guests to talk to the network outside the box hosting the vm hosting the container.</div><div><br></div><div>I am wondering if anyone has had any experience making such a configuration work.</div><div><br></div><div>I'll try to fill in the relevant details.</div><div><br></div><div>The host box has the following network devices:</div><div><br></div><div><div><font face="Courier New">eth0 Link encap:Ethernet HWaddr 00:90:f5:b5:eb:e6 </font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:0 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)</font></div><div><font face="Courier New"> Interrupt:67 </font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">lo Link encap:Local Loopback </font></div><div><font face="Courier New"> inet addr:127.0.0.1 Mask:255.0.0.0</font></div><div><font face="Courier New"> inet6 addr: ::1/128 Scope:Host</font></div><div><font face="Courier New"> UP LOOPBACK RUNNING MTU:16436 Metric:1</font></div><div><font face="Courier New"> RX packets:460 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:460 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:0 </font></div><div><font face="Courier New"> RX bytes:35888 (35.8 KB) TX bytes:35888 (35.8 KB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">vmnet1 Link encap:Ethernet HWaddr 00:50:56:c0:00:01 </font></div><div><font face="Courier New"> inet addr:172.16.67.1 Bcast:172.16.67.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::250:56ff:fec0:1/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:0 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:79 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">vmnet8 Link encap:Ethernet HWaddr 00:50:56:c0:00:08 </font></div><div><font face="Courier New"> inet addr:192.168.59.1 Bcast:192.168.59.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::250:56ff:fec0:8/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:1 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:79 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">wlan0 Link encap:Ethernet HWaddr e0:91:53:35:39:97 </font></div><div><font face="Courier New"> inet addr:192.168.0.100 Bcast:192.168.0.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::e291:53ff:fe35:3997/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:11345 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:10362 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:6413336 (6.4 MB) TX bytes:1866889 (1.8 MB)</font></div><div><font face="Courier New"> Interrupt:18 Memory:ffffc90003320000-ffffc90003320100 </font></div></div><div><br></div><div>wlan0 eventually leads to a wireless router and the internet.</div><div><br></div><div>The routing table on the host is:</div><div><br></div><div><div><font face="Courier New">Kernel IP routing table</font></div><div><font face="Courier New">Destination Gateway Genmask Flags Metric Ref Use Iface</font></div><div><font face="Courier New">192.168.0.0 0.0.0.0 255.255.255.0 U 2 0 0 wlan0</font></div><div><font face="Courier New">172.16.67.0 0.0.0.0 255.255.255.0 U 0 0 0 vmnet1</font></div><div><font face="Courier New">192.168.59.0 0.0.0.0 255.255.255.0 U 0 0 0 vmnet8</font></div><div><font face="Courier New">169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 wlan0</font></div><div><font face="Courier New">0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 wlan0</font></div></div><div><br></div><div>Next, the vm's network devices:</div><div><br></div><div><div><font face="Courier New">br0 Link encap:Ethernet HWaddr 2a:40:9c:29:c8:ac </font></div><div><font face="Courier New"> inet addr:192.168.2.254 Bcast:192.168.2.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::2c4c:89ff:fe79:d51f/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:755 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:809 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:0 </font></div><div><font face="Courier New"> RX bytes:120905 (120.9 KB) TX bytes:83379 (83.3 KB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">eth0 Link encap:Ethernet HWaddr 00:0c:29:17:22:0a </font></div><div><font face="Courier New"> inet addr:192.168.59.128 Bcast:192.168.59.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::20c:29ff:fe17:220a/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:129267 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:75502 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:189241404 (189.2 MB) TX bytes:4219414 (4.2 MB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">lo Link encap:Local Loopback </font></div><div><font face="Courier New"> inet addr:127.0.0.1 Mask:255.0.0.0</font></div><div><font face="Courier New"> inet6 addr: ::1/128 Scope:Host</font></div><div><font face="Courier New"> UP LOOPBACK RUNNING MTU:16436 Metric:1</font></div><div><font face="Courier New"> RX packets:69 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:69 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:0 </font></div><div><font face="Courier New"> RX bytes:7048 (7.0 KB) TX bytes:7048 (7.0 KB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">veth5c5qSm Link encap:Ethernet HWaddr 2a:40:9c:29:c8:ac </font></div><div><font face="Courier New"> inet6 addr: fe80::2840:9cff:fe29:c8ac/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:458 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:592 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:63277 (63.2 KB) TX bytes:59013 (59.0 KB)</font></div></div><div><br></div><div>br0 is the bridge device intended to join all the lxc containers and eth0 connects to the host's vmnet8</div><div><br></div><div>The routing table on the vm is:</div><div><br></div><div><div><font face="Courier New">Kernel IP routing table</font></div><div><font face="Courier New">Destination Gateway Genmask Flags Metric Ref Use Iface</font></div><div><font face="Courier New">0.0.0.0 192.168.59.2 0.0.0.0 UG 100 0 0 eth0</font></div><div><font face="Courier New">192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br0</font></div><div><font face="Courier New">192.168.59.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0</font></div></div><div><br></div><div>This routing table confuses me because the default gateway is the first entry which seems strange. That being said, I can reach the internet from the vm without any problem.</div><div>I've also got iptables configured to do the masquerading for the br0 device:</div><div><br></div><div><div><div><font face="Courier New"># Generated by iptables-save v1.4.12 on Thu Sep 6 21:23:39 2012</font></div><div><font face="Courier New">*nat</font></div><div><font face="Courier New">:PREROUTING ACCEPT [290:47394]</font></div><div><font face="Courier New">:INPUT ACCEPT [13:2468]</font></div><div><font face="Courier New">:OUTPUT ACCEPT [1719:115881]</font></div><div><font face="Courier New">:POSTROUTING ACCEPT [28:2088]</font></div><div><font face="Courier New">:ForwardedPorts - [0:0]</font></div><div><font face="Courier New">-A PREROUTING -j ForwardedPorts</font></div><div><font face="Courier New">-A POSTROUTING -m mark --mark 0x9 -j MASQUERADE</font></div><div><font face="Courier New">-A POSTROUTING -o eth0 -j MASQUERADE</font></div><div><font face="Courier New">COMMIT</font></div><div><font face="Courier New"># Completed on Thu Sep 6 21:23:39 2012</font></div><div><font face="Courier New"># Generated by iptables-save v1.4.12 on Thu Sep 6 21:23:39 2012</font></div><div><font face="Courier New">*mangle</font></div><div><font face="Courier New">:PREROUTING ACCEPT [3:354]</font></div><div><font face="Courier New">:INPUT ACCEPT [3:354]</font></div><div><font face="Courier New">:FORWARD ACCEPT [0:0]</font></div><div><font face="Courier New">:OUTPUT ACCEPT [0:0]</font></div><div><font face="Courier New">:POSTROUTING ACCEPT [0:0]</font></div><div><font face="Courier New">:Masquerade - [0:0]</font></div><div><font face="Courier New">-A PREROUTING -j Masquerade</font></div><div><font face="Courier New">-A Masquerade -i br0 -j MARK --set-xmark 0x9/0xffffffff</font></div><div><font face="Courier New">COMMIT</font></div><div><font face="Courier New"># Completed on Thu Sep 6 21:23:39 2012</font></div><div><font face="Courier New"># Generated by iptables-save v1.4.12 on Thu Sep 6 21:23:39 2012</font></div><div><font face="Courier New">*filter</font></div><div><font face="Courier New">:INPUT ACCEPT [0:0]</font></div><div><font face="Courier New">:FORWARD ACCEPT [0:0]</font></div><div><font face="Courier New">:OUTPUT ACCEPT [77183:3310031]</font></div><div><font face="Courier New">:Blocked - [0:0]</font></div><div><font face="Courier New">:Firewall - [0:0]</font></div><div><font face="Courier New">:RoutedDevices - [0:0]</font></div><div><font face="Courier New">:TrustedDevices - [0:0]</font></div><div><font face="Courier New">:TrustedPorts - [0:0]</font></div><div><font face="Courier New">-A INPUT -j Blocked</font></div><div><font face="Courier New">-A INPUT -j Firewall</font></div><div><font face="Courier New">-A FORWARD -j Blocked</font></div><div><font face="Courier New">-A FORWARD -j RoutedDevices</font></div><div><font face="Courier New">-A FORWARD -j Firewall</font></div><div><font face="Courier New">-A Firewall -j TrustedDevices</font></div><div><font face="Courier New">-A Firewall -p icmp -m icmp --icmp-type any -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -p esp -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -p ah -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -p udp -m udp --dport 631 -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -p tcp -m tcp --dport 631 -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -m state --state RELATED,ESTABLISHED -j ACCEPT</font></div><div><font face="Courier New">-A Firewall -j TrustedPorts</font></div><div><font face="Courier New">-A Firewall -j REJECT --reject-with icmp-host-prohibited</font></div><div><font face="Courier New">-A RoutedDevices -i br0 -j ACCEPT</font></div><div><font face="Courier New">-A TrustedDevices -i lo -j ACCEPT</font></div><div><font face="Courier New">-A TrustedDevices -i br0 -j ACCEPT</font></div><div><font face="Courier New">-A TrustedPorts -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT</font></div><div><font face="Courier New">COMMIT</font></div><div><font face="Courier New"># Completed on Thu Sep 6 21:23:39 2012</font></div></div></div><div><br></div><div>Finally, one of the lxc containers:</div><div><br></div><div><div><font face="Courier New">eth0 Link encap:Ethernet HWaddr 00:16:3e:38:88:bb </font></div><div><font face="Courier New"> inet addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0</font></div><div><font face="Courier New"> inet6 addr: fe80::216:3eff:fe38:88bb/64 Scope:Link</font></div><div><font face="Courier New"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</font></div><div><font face="Courier New"> RX packets:900 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:697 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:1000 </font></div><div><font face="Courier New"> RX bytes:90043 (90.0 KB) TX bytes:93265 (93.2 KB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">lo Link encap:Local Loopback </font></div><div><font face="Courier New"> inet addr:127.0.0.1 Mask:255.0.0.0</font></div><div><font face="Courier New"> inet6 addr: ::1/128 Scope:Host</font></div><div><font face="Courier New"> UP LOOPBACK RUNNING MTU:16436 Metric:1</font></div><div><font face="Courier New"> RX packets:39 errors:0 dropped:0 overruns:0 frame:0</font></div><div><font face="Courier New"> TX packets:39 errors:0 dropped:0 overruns:0 carrier:0</font></div><div><font face="Courier New"> collisions:0 txqueuelen:0 </font></div><div><font face="Courier New"> RX bytes:4292 (4.2 KB) TX bytes:4292 (4.2 KB)</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">Kernel IP routing table</font></div><div><font face="Courier New">Destination Gateway Genmask Flags Metric Ref Use Iface</font></div><div><font face="Courier New">0.0.0.0 192.168.2.1 0.0.0.0 UG 0 0 0 eth0</font></div><div><font face="Courier New">192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0</font></div></div><div><br></div><div>Again the strange routing table with the default gateway at the top of the list. I should mention that it takes a strangely long time (several seconds) to ssh from the vm into one of the containers...</div><div>From the lxc container, I can ping myself (192.168.2.1), my default gateway (192.168.2.254), and my default gateway's outgoing interface (192.168.59.128)</div><div>However, I cannot ping my default gateway's default gateway (192.168.59.2):</div><div><br></div><div><div><font face="Courier New">PING 192.168.59.2 (192.168.59.2) 56(84) bytes of data.</font></div><div><font face="Courier New">From 192.168.2.1 icmp_seq=1 Destination Host Unreachable</font></div><div><font face="Courier New">From 192.168.2.1 icmp_seq=2 Destination Host Unreachable</font></div><div><font face="Courier New">From 192.168.2.1 icmp_seq=3 Destination Host Unreachable</font></div><div><font face="Courier New">From 192.168.2.1 icmp_seq=4 Destination Host Unreachable</font></div><div><font face="Courier New">From 192.168.2.1 icmp_seq=5 Destination Host Unreachable</font></div><div><font face="Courier New"><br></font></div><div><font face="Courier New">--- 192.168.59.2 ping statistics ---</font></div><div><font face="Courier New">5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4022ms</font></div><div><br></div></div><div>Anyway, I would sincerely appreciate any help...</div><div><br></div><div>Cheers,</div><div><br></div><div>Peter-Frank Spierenburg.</div><div><br></div> </div>
</div> </div></body>
</html>