[Lxc-users] Two virtual interfaces in a container

[Nirmal Guhan]

On Fri, Oct 22, 2010 at 3:48 PM, Nirmal Guhan <vavatutu at gmail.com> wrote:
> On Tue, Oct 19, 2010 at 3:58 PM, Serge E. Hallyn
> <serge.hallyn at canonical.com> wrote:
>> Quoting Nirmal Guhan (vavatutu at gmail.com):
>>> On Tue, Oct 19, 2010 at 3:03 PM, Serge E. Hallyn
>>> <serge.hallyn at canonical.com> wrote:
>>> > Quoting Serge E. Hallyn (serge.hallyn at canonical.com):
>>> >> Quoting Nirmal Guhan (vavatutu at gmail.com):
>>> >> > Hi,
>>> >> >
>>> >> > I have a requirement to create two virtual interfaces (eth0, eth1) in
>>> >> > a linux container and separate traffic between the two based on ip
>>> >> > route. Basically eth0 (or eth1) should be used for external world and
>>> >> > eth1 for communication terminating at host. How do I go about doing
>>> >> > this?
>>> >> >
>>> >> > I created two interfaces in the config and can see both of them in the
>>> >> > container.
>>> >> >
>>> >> > lxc.network.type = veth
>>> >> > lxc.network.link = br0
>>> >> > lxc.network.ipv4 = 128.107.159.183/22
>>> >> > lxc.network.name = eth0
>>> >> > lxc.network.flags = up
>>> >> > lxc.network.mtu = 1500
>>> >> > lxc.network.type = veth
>>> >> > lxc.network.link = br0
>>> >>
>>> >> If you want eth1 to be connected internally only, then shouldn't
>>> >> you create a bridge br1, and use that here?  Don't connect br1
>>> >> to the physical nic, and you'll have your host-only bridge.
>>>
>>> Ok. This is what I did.
>>> #brctl addbr br1
>>>
>>> Modified above config to lxc.network.link=br1 for eth1 and removed
>>> eth0 so there is only one i/f. Since br1 is not attached to nic, how
>>> do I now test host<->guest communication.Obviously I can't reach eth0
>>> ip from lxc.
>>
>> Easiest and most telling wrt whether your setup will work, would be
>> to create a second container the same way, and try to ping or
>> nc to each other.
>>
>> -serge
>>
> Thanks. Pinging between containers work. Going back to my original
> query, I need a tap interface as well in the bridge so it is actually
> tap<->bridge<->veth on container . So I created a tap 'gtap' interface
> in the host and added it to br1. Assinged IP to gtap and tried to ping
> from the container but that does not work. Here are some add'l info :
>
> 26: gtap: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc
> pfifo_fast state UNKNOWN qlen 500
>    link/ether fa:ad:bb:c0:d4:4c brd ff:ff:ff:ff:ff:ff
>    inet 192.168.1.15/24 brd 192.168.1.255 scope global gtap
>    inet6 fe80::f8ad:bbff:fec0:d44c/64 scope link
>       valid_lft forever preferred_lft forever
> 27: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
>    link/ether 92:e1:7e:95:4d:bc brd ff:ff:ff:ff:ff:ff
>    inet6 fe80::f8ad:bbff:fec0:d44c/64 scope link
>       valid_lft forever preferred_lft forever
>
> [128:~]$ brctl show
> bridge name     bridge id               STP enabled     interfaces
> br1             8000.92e17e954dbc       no              gtap
>                                                        veths4EgPK
>
> $ ip route show
> 192.168.1.0/24 dev gtap  proto kernel  scope link  src 192.168.1.15
> $sbin/arp
> Address                  HWtype  HWaddress           Flags Mask            Iface
> 192.168.1.10                     (incomplete)                              gtap
>
> From container:
> $ip route show
> 192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.10
> $ /sbin/arp
> Address                  HWtype  HWaddress           Flags Mask            Iface
> 192.168.1.15                     (incomplete)                              eth1
>
> Do I assign IP address to br1 instead of gtap?
>
> Thanks,
> Nirmal
>
Here is an update : After adding a route as
ip route add 192.168.1.0/24 dev br1
I can ping tap interface from container. But two weird things :
1."tcpdump -i gtap" does not show any packet but "tcpdump -i br1"
shows the packets.
2. If I bring down gtap as in "ifconfig gtap down" am still able to
ping gtap ip with the above ip route configured.
Still looking for reasoning...

-Nirmal