[Lxc-users] Two virtual interfaces in a container

[Daniel Lezcano]

On 10/23/2010 11:08 PM, Nirmal Guhan wrote:
> On Fri, Oct 22, 2010 at 3:48 PM, Nirmal Guhan<vavatutu at gmail.com>  wrote:
>> On Tue, Oct 19, 2010 at 3:58 PM, Serge E. Hallyn
>> <serge.hallyn at canonical.com>  wrote:
>>> Quoting Nirmal Guhan (vavatutu at gmail.com):
>>>> On Tue, Oct 19, 2010 at 3:03 PM, Serge E. Hallyn
>>>> <serge.hallyn at canonical.com>  wrote:
>>>>> Quoting Serge E. Hallyn (serge.hallyn at canonical.com):
>>>>>> Quoting Nirmal Guhan (vavatutu at gmail.com):
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have a requirement to create two virtual interfaces (eth0, eth1) in
>>>>>>> a linux container and separate traffic between the two based on ip
>>>>>>> route. Basically eth0 (or eth1) should be used for external world and
>>>>>>> eth1 for communication terminating at host. How do I go about doing
>>>>>>> this?
>>>>>>>
>>>>>>> I created two interfaces in the config and can see both of them in the
>>>>>>> container.
>>>>>>>
>>>>>>> lxc.network.type = veth
>>>>>>> lxc.network.link = br0
>>>>>>> lxc.network.ipv4 = 128.107.159.183/22
>>>>>>> lxc.network.name = eth0
>>>>>>> lxc.network.flags = up
>>>>>>> lxc.network.mtu = 1500
>>>>>>> lxc.network.type = veth
>>>>>>> lxc.network.link = br0
>>>>>>
>>>>>> If you want eth1 to be connected internally only, then shouldn't
>>>>>> you create a bridge br1, and use that here?  Don't connect br1
>>>>>> to the physical nic, and you'll have your host-only bridge.
>>>>
>>>> Ok. This is what I did.
>>>> #brctl addbr br1
>>>>
>>>> Modified above config to lxc.network.link=br1 for eth1 and removed
>>>> eth0 so there is only one i/f. Since br1 is not attached to nic, how
>>>> do I now test host<->guest communication.Obviously I can't reach eth0
>>>> ip from lxc.
>>>
>>> Easiest and most telling wrt whether your setup will work, would be
>>> to create a second container the same way, and try to ping or
>>> nc to each other.
>>>
>>> -serge
>>>
>> Thanks. Pinging between containers work. Going back to my original
>> query, I need a tap interface as well in the bridge so it is actually
>> tap<->bridge<->veth on container . So I created a tap 'gtap' interface
>> in the host and added it to br1. Assinged IP to gtap and tried to ping
>> from the container but that does not work. Here are some add'l info :
>>
>> 26: gtap:<BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP>  mtu 1500 qdisc
>> pfifo_fast state UNKNOWN qlen 500
>>     link/ether fa:ad:bb:c0:d4:4c brd ff:ff:ff:ff:ff:ff
>>     inet 192.168.1.15/24 brd 192.168.1.255 scope global gtap
>>     inet6 fe80::f8ad:bbff:fec0:d44c/64 scope link
>>        valid_lft forever preferred_lft forever
>> 27: br1:<BROADCAST,MULTICAST,UP,LOWER_UP>  mtu 1500 qdisc noqueue state UNKNOWN
>>     link/ether 92:e1:7e:95:4d:bc brd ff:ff:ff:ff:ff:ff
>>     inet6 fe80::f8ad:bbff:fec0:d44c/64 scope link
>>        valid_lft forever preferred_lft forever
>>
>> [128:~]$ brctl show
>> bridge name     bridge id               STP enabled     interfaces
>> br1             8000.92e17e954dbc       no              gtap
>>                                                         veths4EgPK
>>
>> $ ip route show
>> 192.168.1.0/24 dev gtap  proto kernel  scope link  src 192.168.1.15
>> $sbin/arp
>> Address                  HWtype  HWaddress           Flags Mask            Iface
>> 192.168.1.10                     (incomplete)                              gtap
>>
>>  From container:
>> $ip route show
>> 192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.10
>> $ /sbin/arp
>> Address                  HWtype  HWaddress           Flags Mask            Iface
>> 192.168.1.15                     (incomplete)                              eth1
>>
>> Do I assign IP address to br1 instead of gtap?
>>
>> Thanks,
>> Nirmal
>>
> Here is an update : After adding a route as
> ip route add 192.168.1.0/24 dev br1
> I can ping tap interface from container. But two weird things :
> 1."tcpdump -i gtap" does not show any packet but "tcpdump -i br1"
> shows the packets.

That's logical, the bridge is the aggregator of your interfaces. You can 
not assume the interface will act as you expect when it is attached to 
the bridge because the traffic is hooked in the kernel and the packets 
are redirected to the bridge code.

> 2. If I bring down gtap as in "ifconfig gtap down" am still able to
> ping gtap ip with the above ip route configured.
> Still looking for reasoning...

When you assign an IP address to an interfaces that automatically create 
the routes. Assigning IP addresses is a way to automatically create / 
destroy the routes. You can create some routes without an IP address on 
an interface and the packet will reach at least the layer 3 of the 
network stack.

If you want to delete an IP address to your interface you can use ip 
addr del <ip>/<prefix> dev gtap or ifconfig gtap 0.0.0.0