[lxc-users] Fun with lxc.network.type=phys

[Fajar A. Nugraha]

On Wed, Jan 21, 2015 at 10:50 PM, ScrumpyJack <scrumpyjack at me.com> wrote:
> On Wed, 21 Jan 2015, Fajar A. Nugraha wrote:
>
>> On Wed, Jan 21, 2015 at 7:09 PM, scrumpyjack <scrumpyjack at me.com> wrote:
>>
>> > Yes, i want to give a /32 to a container.

>> Relevant part of container's /etc/network/interfaces
>> ###
>> auto eth0
>> iface eth0 inet static
>>         address 100.0.0.11
>>         netmask 255.255.255.255
>> # force route for host's br0
>> up ip route add 192.168.124.1 dev eth0
>> # ... and use it for default route
>> up ip route add default via 192.168.124.1
>
> Yup, thanks, this worked for me.
> I was trying to use macvlan and phys to avoid having to add my eth0 to the
> bridgeport and putting into promiscuois mode, which this solves


There's actually another alternative which does NOT involve bridging
altogether, but requires an additional script for each container. It
routes container traffic directly thru host side of container's veth
interface, and use host's IP for container's default route.

Note that you can actually use ANY IP that is in the host but not in
the container. For example, since by default lxc creates a bridge
called lxcbr0 with IP address 10.0.3.1, you can use that for
container's default route, even when the container interface is not
bridged to lxcbr0. Or you can use the host's public IP (which is what
I used in this example).

Something like this:

Relevant part of host's /etc/network/interfaces
###
auto eth0
iface eth0 inet static
address 100.0.0.10
netmask 255.255.255.0
gateway 100.0.0.1
# this part functions similar as proxy arp, force eth0 to accepts packets
# destined for the container's IP using static arp
up arp -i eth0 -Ds 100.0.0.11 eth0 pub || true

# note that you do NOT need any bridge in this setup
###


Relevant part of container config. Note that this only sets persistent
vif mac & name, but does NOT set any bridge.
###
lxc.network.type=veth
# in this setup you do NOT specify any bridge for lxc.network.link
#lxc.network.link=
lxc.network.veth.pair=veth-c1-0
lxc.network.flags=up
lxc.network.hwaddr = 00:16:3E:FD:46:25
# this script will add specific route
lxc.network.script.up = /path/to/your/container/network_up_script
###


Content of /path/to/your/container/network_up_script (put it wherever
you want, /var/lib/container_name/network_up.sh is a good place since
each container will need its own script)
###
#!/bin/bash
# set specific route directly thru the host's side of container's veth interface
/sbin/ip route add 100.0.0.11/32 dev veth-c1-0
###


Relevant part of container's /etc/network/interfaces
###
auto eth0
iface eth0 inet static
        address 100.0.0.11
        netmask 255.255.255.255
# force route for host's IP
up ip route add 100.0.0.10 dev eth0
# ... and use it for default route
up ip route add default via 100.0.0.10
###

-- 
Fajar