[Lxc-users] Networking fails sporadically in containers on ubuntu server 12.10
David Parks
davidparks21 at yahoo.com
Wed Apr 3 06:27:06 UTC 2013
I take it back, I'm still reproducing the issue. I just stopped 5 LXC
containers, cloned 5 from a base image, and re-set the IP to the same as the
original 5 were using (e.g. the MAC changed as I don't have that defined in
the config file). Restarting the first one now and it won't ping the router
or get out to the internet, and there's a ~30 sec delay SSHing to it from
the host. At this same time I have one other LXC container running that
works (this one starting working after I changed the MAC of the NAT
interface in virtualbox, eth0 on the host OS).
From: David Parks [mailto:davidparks21 at yahoo.com]
Sent: Wednesday, April 03, 2013 12:13 PM
To: 'lxc-users at lists.sourceforge.net'
Subject: RE: [Lxc-users] Networking fails sporadically in containers on
ubuntu server 12.10
I should also add to this that I just changed the MAC address of the
virtualbox NAT interface to 22:00:22:00:22:00 as Serge had previously
suggested. It's working now, though I can be confident that changing the
virtualbox NAT's MAC was the solution or if it's just intermittently working
now. I'll make another update in a few days if it continues to work, or if
it fails again.
Thanks, by the way, for the super help!!
Dave
From: David Parks [mailto:davidparks21 at yahoo.com]
Sent: Wednesday, April 03, 2013 11:38 AM
To: lxc-users at lists.sourceforge.net
Subject: Re: [Lxc-users] Networking fails sporadically in containers on
ubuntu server 12.10
Yesterday I couldn't reproduce the issue with 2 LXC containers running, but
today I've rebooted everything and I have one LXC container running and it
can't access the router or internet (but can access the host IP and other
LXC containers).
Right now the br0 MAC is configured to be the same as the eth0 MAC address.
Is it odd that, on the host OS, when I check the ARP table the mac address
for the LXC IP (10.1.3.45) is seen as the MAC *inside* the container, not of
vethR1s1ny on the host. But maybe that's expected behavior?
Also notable, from the host OS I can ping 10.1.3.45 (the LXC containers IP),
but when I SSH to it there is around a 30 second delay before the SSH login
prompt comes up. When things are working there is no such lag.
I turned on debug logging like Serge suggested, but don't see anything
interesting in there (also below).
David
IFCONFIG / BRCTL SHOW / ARP -A
davidparks21 at ubuntuserver:~$ ifconfig
br0 Link encap:Ethernet HWaddr 08:00:27:ca:5f:7a
inet addr:10.1.0.5 Bcast:10.1.255.255 Mask:255.255.0.0
inet6 addr: fe80::a00:27ff:feca:5f7a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:983 errors:0 dropped:0 overruns:0 frame:0
TX packets:949 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:87727 (87.7 KB) TX bytes:108287 (108.2 KB)
eth0 Link encap:Ethernet HWaddr 08:00:27:ca:5f:7a
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:883 errors:0 dropped:0 overruns:0 frame:0
TX packets:725 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:97519 (97.5 KB) TX bytes:79510 (79.5 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:616 (616.0 B) TX bytes:616 (616.0 B)
vethR1s1ny Link encap:Ethernet HWaddr fe:d5:4a:f2:07:db
inet6 addr: fe80::fcd5:4aff:fef2:7db/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:247 errors:0 dropped:0 overruns:0 frame:0
TX packets:367 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:28718 (28.7 KB) TX bytes:37972 (37.9 KB)
davidparks21 at ubuntuserver:~$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.080027ca5f7a no eth0
vethR1s1ny
davidparks21 at ubuntuserver:~$ arp -a
? (10.1.0.2) at 52:54:00:12:35:02 [ether] on br0
? (10.1.3.45) at 26:b2:18:b5:fc:e7 [ether] on br0
LXC DEBUG LOG
davidparks21 at ubuntuserver:~/lxc$ cat lxc.debug
lxc-start 1364962992.454 INFO lxc_conf - tty's configured
lxc-start 1364962992.454 INFO lxc_start - 'cloudera-manager' is
initialized
lxc-start 1364962992.560 INFO lxc_conf - opened
/var/lib/lxc/cloudera-manager/rootfs.hold as fd 18
lxc-start 1364962992.562 INFO lxc_cgroup - [1] found cgroup
mounted at '/sys/fs/cgroup/cpuset',opts='rw,relatime,cpuset,clone_children'
lxc-start 1364962992.733 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/cpuset//lxc/cloudera-manager'
lxc-start 1364962992.733 INFO lxc_cgroup - [2] found cgroup
mounted at '/sys/fs/cgroup/cpu',opts='rw,relatime,cpu,clone_children'
lxc-start 1364962992.743 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/cpu//lxc/cloudera-manager'
lxc-start 1364962992.743 INFO lxc_cgroup - [3] found cgroup
mounted at
'/sys/fs/cgroup/cpuacct',opts='rw,relatime,cpuacct,clone_children'
lxc-start 1364962992.749 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/cpuacct//lxc/cloudera-manager'
lxc-start 1364962992.749 INFO lxc_cgroup - [4] found cgroup
mounted at '/sys/fs/cgroup/memory',opts='rw,relatime,memory,clone_children'
lxc-start 1364962992.757 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/memory//lxc/cloudera-manager'
lxc-start 1364962992.757 INFO lxc_cgroup - [5] found cgroup
mounted at
'/sys/fs/cgroup/devices',opts='rw,relatime,devices,clone_children'
lxc-start 1364962992.769 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/devices//lxc/cloudera-manager'
lxc-start 1364962992.769 INFO lxc_cgroup - [6] found cgroup
mounted at
'/sys/fs/cgroup/freezer',opts='rw,relatime,freezer,clone_children'
lxc-start 1364962992.770 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/freezer//lxc/cloudera-manager'
lxc-start 1364962992.770 INFO lxc_cgroup - [7] found cgroup
mounted at '/sys/fs/cgroup/blkio',opts='rw,relatime,blkio,clone_children'
lxc-start 1364962992.771 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/blkio//lxc/cloudera-manager'
lxc-start 1364962992.771 INFO lxc_cgroup - [8] found cgroup
mounted at
'/sys/fs/cgroup/perf_event',opts='rw,relatime,perf_event,clone_children'
lxc-start 1364962992.772 INFO lxc_cgroup - created cgroup
'/sys/fs/cgroup/perf_event//lxc/cloudera-manager'
lxc-start 1364962992.774 INFO lxc_conf - 'cloudera-manager'
hostname has been setup
lxc-start 1364962992.778 INFO lxc_conf - network has been setup
lxc-start 1364962992.778 INFO lxc_conf - mount points have been
setup
lxc-start 1364962992.778 INFO lxc_conf - mount points have been
setup
lxc-start 1364962992.781 INFO lxc_conf - cgroup has been setup
lxc-start 1364962992.781 INFO lxc_conf - created
/usr/lib/x86_64-linux-gnu/lxc/dev/lxc
lxc-start 1364962992.781 INFO lxc_conf - console has been setup on
lxc/console
lxc-start 1364962992.781 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1364962992.781 INFO lxc_conf - rootfs path is
./var/lib/lxc/cloudera-manager/rootfs., mount is
./usr/lib/x86_64-linux-gnu/lxc.
lxc-start 1364962992.781 INFO lxc_conf - I am 1, /proc/self points
to 1
lxc-start 1364962992.782 INFO lxc_conf - created new pts instance
lxc-start 1364962992.782 INFO lxc_conf - set personality to '0x0'
lxc-start 1364962992.782 NOTICE lxc_conf - 'cloudera-manager' is
setup.
lxc-start 1364962992.782 INFO lxc_start - changed apparmor profile
to lxc-container-default
lxc-start 1364962992.782 NOTICE lxc_start - exec'ing '/sbin/init'
lxc-start 1364962992.783 NOTICE lxc_start - '/sbin/init' started
with pid '3333'
lxc-start 1364962992.783 WARN lxc_console - console input disabled
lxc-start 1364962992.783 WARN lxc_start - invalid pid for SIGCHLD
-----Original Message-----
From: Hans Feldt [mailto:hans.feldt at ericsson.com]
Sent: Tuesday, April 02, 2013 9:46 PM
To: Serge Hallyn
Cc: David Parks; lxc-users at lists.sourceforge.net
Subject: Re: [Lxc-users] Networking fails sporadically in containers on
ubuntu server 12.10
Could it be the issue described in
<http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-
ports/>
http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-p
orts/
?
I encountered that one and solved it by creating a dummy interface with the
same life cycle as the bridge.
thanks,
Hans
On 04/01/2013 02:34 PM, Serge Hallyn wrote:
> Quoting David Parks ( <mailto:davidparks21 at yahoo.com>
davidparks21 at yahoo.com):
>> davidparks21 at hostOS:~$ arp -a
>>
>> ? (10.1.0.4) at c6:63:bb:a7:d8:60 [ether] on br0
>>
>> ? (10.1.0.2) at 52:54:00:12:35:02 [ether] on br0
>>
>> ? (10.1.0.45) at 32:8c:fc:c1:7f:e5 [ether] on br0
>
> Can you set the macaddr on your virtualbox nic to something lower,
> like something starting with 22: ?
>
> 32: sounds really low for a veth peer nic, I'm surprised - mine are
> always fe:. In fact, looking at
> src/lxc/conf.c:setup_private_host_hw_addr(), it makes sure to set 0xfe
> in the first two bytes. So why are you getting 32: ? Hmmm.
>
> Could you also try adding '-l info -o lxc.debug' to your lxc-start
> command on a failing container, and send the resulting lxc.debug file?
> We're looking for output from:
>
> DEBUG("mac address of host interface '%s' changed to private "
> "%02x:%02x:%02x:%02x:%02x:%02x", veth1,
>
> thanks,
> -serge
>
> ----------------------------------------------------------------------
> -------- Own the Future-Intel® Level Up Game Demo Contest 2013
> Rise to greatness in Intel's independent game demo contest.
> Compete for recognition, cash, and the chance to get your game on
> Steam. $5K grand prize plus 10 genre and skill prizes.
> Submit your demo by 6/6/13. <http://p.sf.net/sfu/intel_levelupd2d>
http://p.sf.net/sfu/intel_levelupd2d
> _______________________________________________
> Lxc-users mailing list
> <mailto:Lxc-users at lists.sourceforge.net> Lxc-users at lists.sourceforge.net
> <https://lists.sourceforge.net/lists/listinfo/lxc-users>
https://lists.sourceforge.net/lists/listinfo/lxc-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20130403/63113485/attachment.html>
More information about the lxc-users
mailing list