[Lxc-users] Many containers and too many open files

[Daniel Lezcano]

On 03/02/2011 02:46 PM, Andre Nathan wrote:
> On Wed, 2011-03-02 at 14:24 +0100, Daniel Lezcano wrote:
>>> I could paste my configuration files if you think it'd help you
>>> reproducing the issue.
>> Yes, please :)
> Ok. The test host has a br0 interface which is not attached to any
> physical interface:
>
>    auto br0
>    iface br0 inet static
>      address 192.168.0.1
>      netmask 255.255.0.0
>      broadcast 192.168.255.255
>      bridge_stp off
>      bridge_maxwait 5
>      pre-up /usr/sbin/brctl addbr br0
>      post-up /usr/sbin/brctl setfd br0 0
>      post-down /usr/sbin/brctl delbr br0
>
> I use NAT for container access, translating to the host's eth0 address.
> There is also a MARK rule that I use for bandwidth limiting. These
> commands are run on the host startup:
>
> iptables -t mangle -A PREROUTING -i br0 -j MARK --set-mark 2
> iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source $ETH0_IP
> iptables -P FORWARD DROP
> iptables -A FORWARD -i br0 -o eth0 -j ACCEPT
> iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
> tc qdisc add dev eth0 root handle 1: htb
>
> I'm using a custom container creation script based on the ubuntu
> templace that you can find here:
>
>    http://andre.people.digirati.com.br/lxc-create.sh
>
> It sets up the bandwidth limit for each container and populates the
> container's rootfs (there is a usage message :). It creates
> configuration files like this:
>
>    lxc.utsname = c2
>
>    lxc.network.type = veth
>    lxc.network.link = br0
>    lxc.network.flags = up
>    lxc.network.ipv4 = 192.168.0.2/16 192.168.255.255
>    lxc.network.name = eth0
>    lxc.network.veth.pair = veth0.2
>
>    lxc.tty = 4
>    lxc.pts = 1024
>    lxc.rootfs = /var/lib/lxc/c2/rootfs
>    lxc.mount  = /var/lib/lxc/c2/fstab
>
>    lxc.cgroup.devices.deny = a
>    # /dev/null and zero
>    lxc.cgroup.devices.allow = c 1:3 rwm
>    lxc.cgroup.devices.allow = c 1:5 rwm
>    # consoles
>    lxc.cgroup.devices.allow = c 5:1 rwm
>    lxc.cgroup.devices.allow = c 5:0 rwm
>    #lxc.cgroup.devices.allow = c 4:0 rwm
>    #lxc.cgroup.devices.allow = c 4:1 rwm
>    # /dev/{,u}random
>    lxc.cgroup.devices.allow = c 1:9 rwm
>    lxc.cgroup.devices.allow = c 1:8 rwm
>    lxc.cgroup.devices.allow = c 136:* rwm
>    lxc.cgroup.devices.allow = c 5:2 rwm
>    # rtc
>    lxc.cgroup.devices.allow = c 254:0 rwm
>
>    # capabilities
>    lxc.cap.drop = audit_control audit_write fsetid kill ipc_lock
> ipc_owner lease linux_immutable mac_admin mac_override net_bind_service
> mknod setfcap setpcap sys_admin sys_boot sys_module sys_nice sys_pacct
> sys_ptrace sys_rawio sys_resource sys_time sys_tty_config
>
> and fstab like this:
>
>    /bin /var/lib/lxc/c2/rootfs/bin ext4 bind,ro 0 0
>    /lib /var/lib/lxc/c2/rootfs/lib ext4 bind,ro 0 0
>    /lib64 /var/lib/lxc/c2/rootfs/lib64 ext4 bind,ro 0 0
>    /sbin /var/lib/lxc/c2/rootfs/sbin ext4 bind,ro 0 0
>    /usr /var/lib/lxc/c2/rootfs/usr ext4 bind,ro 0 0
>    /etc/environment /var/lib/lxc/c2/rootfs/etc/environment none bind,ro 0
> 0
>    /etc/resolv.conf /var/lib/lxc/c2/rootfs/etc/resolv.conf none bind,ro 0
> 0
>    /etc/localtime /var/lib/lxc/c2/rootfs/etc/localtime none bind,ro 0 0
>    /etc/network/if-down.d /var/lib/lxc/c2/rootfs/etc/network/if-down.d
> none bind,ro 0 0
>    /etc/network/if-post-down.d /var/lib/lxc/c2/rootfs/etc/network/if-post-down.d none bind,ro 0 0
>    /etc/network/if-pre-up.d /var/lib/lxc/c2/rootfs/etc/network/if-pre-up.d none bind,ro 0 0
>    /etc/network/if-up.d /var/lib/lxc/c2/rootfs/etc/network/if-up.d none
> bind,ro 0 0
>    /etc/login.defs /var/lib/lxc/c2/rootfs/etc/login.defs none bind,ro 0 0
>    /etc/securetty /var/lib/lxc/c2/rootfs/etc/securetty none bind,ro 0 0
>    /etc/pam.conf /var/lib/lxc/c2/rootfs/etc/pam.conf none bind,ro 0 0
>    /etc/pam.d /var/lib/lxc/c2/rootfs/etc/pam.d none bind,ro 0 0
>    /etc/security /var/lib/lxc/c2/rootfs/etc/security none bind,ro 0 0
>    /etc/alternatives /var/lib/lxc/c2/rootfs/etc/alternatives none bind,ro
> 0 0
>    proc /var/lib/lxc/c2/rootfs/proc proc ro,nodev,noexec,nosuid 0 0
>    devpts /var/lib/lxc/c2/rootfs/dev/pts devpts defaults 0 0
>    sysfs /var/lib/lxc/c2/rootfs/sys sysfs defaults 0 0
>
>
> I think that's all. If you need any more info feel free to ask :)

Thanks Andre !