[Lxc-users] Fwd: Container inside an ESX VM

Fri Apr 15 14:24:31 UTC 2011

Missed the list in copy. sorry.

On Fri, Apr 15, 2011 at 3:20 PM, Serge Hallyn <serge.hallyn at canonical.com>wrote:

> Quoting Mauras Olivier (oliver.mauras at gmail.com):
> > Hello,
> >
> > I'm struggling for two days now with some completely weird network
> > behaviours.
> > My host is a virtual machine hosted on an ESX farm. I planned to deploy
> > several containers on it to achieve various tasks.
> >
> > Host is running Scientific Linux 6 with default kernel (2.6.32), and my
> > container is an Oracle Linux 6. I discovered that i had to change ESX
> > vswitch settings to allow promiscuous mode in order to make the host
> bridge
> > correctly behave, but it still gives me weird results.
> > Most of the time after having started the container, network inside the
> > container is erratic. I can ping or ssh from the host to the container,
> but
> > nothing gets out of the container or in the container from the LAN. While
> > the container is still running, if i issue a network restart on the host,
> > the container start behaving correctly and network works again as
> expected.
> > The problem is that it's not reliable at all. If i stop/restart the
> > container several times, it starts losing network again that i can only
> get
> > back by issuing the network restart on the host...
>
> Just a thought, advised by previous libvirt troubles.
>
> Can you look at the mac addresses on the VMWare guest?  Check that the
> eth0 on the vmware guest (i.e. container host) is always lower than
> that of the veths in the guests.
>
> -serge
>
Hi Serge,

Thanks for your reply. This is one thing i found while searching for a
solution and after verifying, everything seems ok.
I did decide by the way to force a high MAC for containers so now they have
fe:ff:xxxxxx MACs. Is there a way to force the veth MAC on the host side?
What i noted is that in any case, when i issue the network restart on the
host and get back network in the container, there's not a change on MACs.
veth stays with the same MAC as long as i don't restart the container.

Here's an example:

[root at host 16:08:39 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP qlen 1000
    link/ether 00:50:56:ae:00:41 brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
    link/ether 00:50:56:ae:00:41 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br0
17: veth56nN7a: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UP qlen 1000
    link/ether a2:8a:f2:ec:f1:1b brd ff:ff:ff:ff:ff:ff

[root at host 16:08:42 ~]# ssh guest

root at guest 's password:

[root at guest ~]# curl -I www.coredumb.net
curl: (7) couldn't connect to host

[root at guest ~]# logout
Connection to guest closed.

[root at host 16:11:22 ~]# /etc/init.d/network
restart

Shutting down interface br0:                               [  OK  ]
Shutting down interface eth0:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:                                [  OK  ]
Bringing up interface br0:                                 [  OK  ]

[root at host 16:11:58 ~]# ip
addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP qlen 1000
    link/ether 00:50:56:ae:00:41 brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state
UNKNOWN
    link/ether 00:50:56:ae:00:41 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br0
17: veth56nN7a: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
pfifo_fast state UP qlen 1000
    link/ether a2:8a:f2:ec:f1:1b brd ff:ff:ff:ff:ff:ff

[root at host 16:12:01 ~]# ssh guest

root at guest's password:

[root at guest ~]# curl -I www.coredumb.net

HTTP/1.1 200 OK
Date: Fri, 15 Apr 2011 14:13:04 GMT
Server: Apache/2.2.15 (Unix) mod_ssl/2.2.15 OpenSSL/0.9.8m DAV/2 PHP/5.2.10
SVN/1.6.4
X-Powered-By: PHP/5.2.10
Expires: Tue, 01 Jan 2002 00:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Type: text/html; charset=ISO-8859-1;
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Set-Cookie: PHPSESSID=giplm9rrr0nmb1fo03f6u39435; path=/

As you see in this example, before issuing the network restart, my veth MAC
was already higher than the eth0 MAC but the guest hadn't a working network
connection.
After restarting network on the host while the guest is still running, as
you can see my MACs haven't changed a bit but now the network inside the
guest is working correctly.

Hope this helps to better understand the problem.

Cheers,
Olivier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20110415/c93ce724/attachment.html>