[Lxc-users] Bug discussion: implementing high virtual device MAC addresses

[Derek Simkowiak]

     There is a behavior in the Linux kernel which can cause a bridge 
device to change MAC address, thus causing a network blackout of several 
seconds (while everybody ARPs the new MAC address flushes the old one).  
This happens when bridging an enslaved interface, like we do with LXC.

     The symptom is that the LXC host will black out for several seconds 
when starting or stopping an LXC container.  Your SSH terminal on the 
host will freeze and become unresponsive.  (It is a random symptom, 
because the blackout only happens if the randomly-assigned MAC address 
of the virtual device is lower than that of the physical eth0 device).

     This behavior was first observed by the libvirt folks when creating 
virtual machines.  You can read more details about it (and how they 
fixed it) here:

https://www.redhat.com/archives/libvir-list/2010-July/msg00450.html
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/584048

     I have observed the symptom under LXC, and the workaround for it 
has been independently confirmed for LXC in this bug report (ID: 3411497):

http://sourceforge.net/tracker/index.php?func=detail&aid=3411497&group_id=163076&atid=826303


     The workaround for the bug is to give the virtual device a high MAC 
address, thus discouraging the bridge device from adapting its MAC 
address as its own.

     I have mentioned this bug on the list before, however, I was 
confused about which MAC address was causing the problem.  This is NOT 
the mac address specified in lxc.conf, like this:

lxc.network.hwaddr = fe:16:3e:fd:5a:5b

     That MAC address has nothing to do with the bug; the host's bridge 
device (br0) will never assume a configured LXC MAC address as its own.  
Instead, the MAC address in question is the one of the virtual vethXXXX 
device, as shown with "ifconfig" on the host:

veth0IEDlk Link encap:Ethernet  HWaddr 4e:34:7c:dc:92:e8
[...snip...]

     That HWaddr should be given a high prefix to avoid the network 
blackouts, just like they've done for libvirt.  That does not exist in 
any config file anywhere; it must be fixed in the LXC source code.

     I looked in network.c for the LXC source code and I think the fix 
should go in lxc_bridge_attach() near line 991.  The fix would put a 
manually-generated MAC address -- one with a high prefix -- into 
ifr.ifr_hwaddr.sa_data and thus replace the random one assigned by the 
kernel.

     However, I'm new to the LXC source and would like some input and 
analysis from a more seasoned contributor.  I would be happy to test and 
maybe even contribute a patch, but I'd like some feedback first.


Thank You,
Derek Simkowiak