[Lxc-users] Many containers and too many open files

[Gary Ballantyne]

Hi

Going back through the list, I couldn't find whether this has been resolved.

I had a similar problem today with a little over 40 containers:

# lxc-start -n gary
lxc-start: Too many open files - failed to inotify_init
lxc-start: failed to add utmp handler to mainloop
lxc-start: mainloop exited with an error
lxc-start: Device or resource busy - failed to remove cgroup '/cgroup/gary'

(On Ubuntu 10.10 (EC2), LXC 0.7.2. Installed with this recipe: 
http://www.phenona.com/blog/using-lxc-linux-containers-in-amazon-ec2/ )

Appreciate your thoughts.

Cheers,

Gary

> On 03/02/2011 02:46 PM, Andre Nathan wrote:
> >  On Wed, 2011-03-02 at 14:24 +0100, Daniel Lezcano wrote:
> >>>  I could paste my configuration files if you think it'd help you
> >>>  reproducing the issue.
> >>  Yes, please :)
> >  Ok. The test host has a br0 interface which is not attached to any
> >  physical interface:
> >
> >     auto br0
> >     iface br0 inet static
> >       address 192.168.0.1
> >       netmask 255.255.0.0
> >       broadcast 192.168.255.255
> >       bridge_stp off
> >       bridge_maxwait 5
> >       pre-up /usr/sbin/brctl addbr br0
> >       post-up /usr/sbin/brctl setfd br0 0
> >       post-down /usr/sbin/brctl delbr br0
> >
> >  I use NAT for container access, translating to the host's eth0 address.
> >  There is also a MARK rule that I use for bandwidth limiting. These
> >  commands are run on the host startup:
> >
> >  iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 2
> >  iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source $ETH0_IP
> >  iptables -P FORWARD DROP
> >  iptables -A FORWARD -i br0 -o eth0 -j ACCEPT
> >  iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
> >  tc qdisc add dev eth0 root handle 1: htb
> >
> >  I'm using a custom container creation script based on the ubuntu
> >  templace that you can find here:
> >
> >     http://andre.people.digirati.com.br/lxc-create.sh
> >
> >  It sets up the bandwidth limit for each container and populates the
> >  container's rootfs (there is a usage message :). It creates
> >  configuration files like this:
> >
> >     lxc.utsname = c2
> >
> >     lxc.network.type = veth
> >     lxc.network.link = br0
> >     lxc.network.flags = up
> >     lxc.network.ipv4 = 192.168.0.2/16 192.168.255.255
> >     lxc.network.name = eth0
> >     lxc.network.veth.pair = veth0.2
> >
> >     lxc.tty = 4
> >     lxc.pts = 1024
> >     lxc.rootfs = /var/lib/lxc/c2/rootfs
> >     lxc.mount  = /var/lib/lxc/c2/fstab
> >
> >     lxc.cgroup.devices.deny = a
> >     # /dev/null and zero
> >     lxc.cgroup.devices.allow = c 1:3 rwm
> >     lxc.cgroup.devices.allow = c 1:5 rwm
> >     # consoles
> >     lxc.cgroup.devices.allow = c 5:1 rwm
> >     lxc.cgroup.devices.allow = c 5:0 rwm
> >     #lxc.cgroup.devices.allow = c 4:0 rwm
> >     #lxc.cgroup.devices.allow = c 4:1 rwm
> >     #//dev//{,u}random
> >     lxc.cgroup.devices.allow = c 1:9 rwm
> >     lxc.cgroup.devices.allow = c 1:8 rwm
> >     lxc.cgroup.devices.allow = c 136:* rwm
> >     lxc.cgroup.devices.allow = c 5:2 rwm
> >     # rtc
> >     lxc.cgroup.devices.allow = c 254:0 rwm
> >
> >     # capabilities
> >     lxc.cap.drop = audit_control audit_write fsetid kill ipc_lock
> >  ipc_owner lease linux_immutable mac_admin mac_override net_bind_service
> >  mknod setfcap setpcap sys_admin sys_boot sys_module sys_nice sys_pacct
> >  sys_ptrace sys_rawio sys_resource sys_time sys_tty_config
> >
> >  and fstab like this:
> >
> >     /bin /var/lib/lxc/c2/rootfs/bin ext4 bind,ro 0 0
> >     /lib /var/lib/lxc/c2/rootfs/lib ext4 bind,ro 0 0
> >     /lib64 /var/lib/lxc/c2/rootfs/lib64 ext4 bind,ro 0 0
> >     /sbin /var/lib/lxc/c2/rootfs/sbin ext4 bind,ro 0 0
> >     /usr /var/lib/lxc/c2/rootfs/usr ext4 bind,ro 0 0
> >     /etc/environment /var/lib/lxc/c2/rootfs/etc/environment none bind,ro 0
> >  0
> >     /etc/resolv.conf /var/lib/lxc/c2/rootfs/etc/resolv.conf none bind,ro 0
> >  0
> >     /etc/localtime /var/lib/lxc/c2/rootfs/etc/localtime none bind,ro 0 0
> >     /etc/network/if-down.d /var/lib/lxc/c2/rootfs/etc/network/if-down.d
> >  none bind,ro 0 0
> >     /etc/network/if-post-down.d /var/lib/lxc/c2/rootfs/etc/network/if-post-down.d none bind,ro 0 0
> >     /etc/network/if-pre-up.d /var/lib/lxc/c2/rootfs/etc/network/if-pre-up.d none bind,ro 0 0
> >     /etc/network/if-up.d /var/lib/lxc/c2/rootfs/etc/network/if-up.d none
> >  bind,ro 0 0
> >     /etc/login.defs /var/lib/lxc/c2/rootfs/etc/login.defs none bind,ro 0 0
> >     /etc/securetty /var/lib/lxc/c2/rootfs/etc/securetty none bind,ro 0 0
> >     /etc/pam.conf /var/lib/lxc/c2/rootfs/etc/pam.conf none bind,ro 0 0
> >     /etc/pam.d /var/lib/lxc/c2/rootfs/etc/pam.d none bind,ro 0 0
> >     /etc/security /var/lib/lxc/c2/rootfs/etc/security none bind,ro 0 0
> >     /etc/alternatives /var/lib/lxc/c2/rootfs/etc/alternatives none bind,ro
> >  0 0
> >     proc /var/lib/lxc/c2/rootfs/proc proc ro,nodev,noexec,nosuid 0 0
> >     devpts /var/lib/lxc/c2/rootfs/dev/pts devpts defaults 0 0
> >     sysfs /var/lib/lxc/c2/rootfs/sys sysfs defaults 0 0
> >
> >
> >  I think that's all. If you need any more info feel free to ask :)
>
> Thanks Andre !

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20110815/a4470851/attachment.html>