[lxc-users] Limit network bandwidth to LXC containers

[Angel Lopez]

Hi,

I need to limit the network bandwidth available to each LXC container using
cgroup's net_cls.classid feature. Each LXC container would have its own
classid value in such a way that all packets from containers would be
tagged with the classid and afterwards classified in the correct host
configured traffic class where the bandwidht limit applies.

To achieve this, I followed these steps:

1. Configure traffic control:

# tc qdisc del dev eno54 root
# tc qdisc add dev eno54 root handle 10: htb
# tc class add dev eno54 parent 10: classid 10:1 htb rate 10mbit
# tc class add dev eno54 parent 10: classid 10:2 htb rate 50mbit
# tc filter add dev eno54 parent 10: protocol ip handle 1: cgroup

The device eno54 is the physical network interface that connect the host
with the network. It's part of the bridge where container virtual network
interfaces are added.

# brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.00163ee2fda2       no              eno54

2. Set the classid value in container config file.

lxctest1 container config file has: lxc.cgroup.net_cls.classid = 0x00100001
lxctest2 container config file has: lxc.cgroup.net_cls.classid = 0x00100002

3. Start both containers. Check that classid is correct and that they
belong to the bridge.

# lxc-start -n lxctest1
# lxc-start -n lxctest2

# cat /sys/fs/cgroup/net_cls/lxc/lxctest1/net_cls.classid
1048577
# cat /sys/fs/cgroup/net_cls/lxc/lxctest2/net_cls.classid
1048578

# brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.00163ee2fda2       no              eno54
                                                        veth0-lxctest1
                                                        veth0-lxctest2
4. Start iperf in both containers.

Expected behaviour: iperf running on container lxctest1 being limited to 10
Mbps and iperf running on lxctest2 container being limited to 50 Mbps.
What I get: both iperf running unconstrained at maximum speed.

5. I took the iperf process running on lxctest1 container and checked that
it was in the tasks of the cgroup

# pstree -c -p 37108
lxc-start(37108)───systemd(37118)─┬─agetty(37167)
                                  ├─agetty(37168)
                                  ├─dbus-daemon(37157)
                                  ├─rsyslogd(37156)─┬─{rsyslogd}(37161)
                                  │                 └─{rsyslogd}(37162)
                                  ├─sshd(37336)───sshd(41156)───
bash(41167)───iperf3(41523)
                                  ├─systemd-journal(37131)
                                  └─systemd-logind(37153)

# cat /sys/fs/cgroup/net_cls/lxc/lxctest1/tasks
37118
37131
37153
37156
37157
37161
37162
37167
37168
37336
39618
41156
41167
41523

# cat /proc/41523/cgroup
10:memory:/lxc/lxctest1
9:hugetlb:/lxc/lxctest1
8:perf_event:/lxc/lxctest1
7:cpuset:/lxc/lxctest1
6:devices:/lxc/lxctest1
5:net_cls,net_prio:/lxc/lxctest1
4:blkio:/lxc/lxctest1
3:cpu,cpuacct:/lxc/lxctest1
2:freezer:/lxc/lxctest1
1:name=systemd:/user.slice/user-0.slice/session-1288.
scope/user.slice/user-0.slice/session-1288.scope

6. I don't know how to check that packets going out the container are
actually being tagged with the classid value, but the reality is that
packets are not filtered acording this value on the host and are not going
to the correct class, where bandwidth limit is applied.

7. I'm using Oracle Linux 7 and the standard lxc package delivered in this
distribution. Versions:

# uname -a
Linux exapru-aa.dit.aeat 4.1.12-112.14.15.el7uek.x86_64 #2 SMP Thu Feb 8
09:58:19 PST 2018 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/oracle-release
Oracle Linux Server release 7.4

# yum info lxc
Loaded plugins: ulninfo
Installed Packages
Name        : lxc
Arch        : x86_64
Version     : 1.1.5
Release     : 2.0.9.el7
Size        : 725 k
Repo        : installed
>From repo   : ol7_latest
Summary     : Linux Containers userspace tools
URL         : http://linuxcontainers.org
License     : LGPLv2+
Description : Containers are insulated areas inside a system, which have
their own namespace
            : for filesystem, network, PID, IPC, CPU and memory allocation
and which can be
            : created using the Control Group and Namespace features
included in the Linux
            : kernel.
            :
            : This package provides the lxc-* tools, which can be used to
start a single
            : daemon in a container, or to boot an entire "containerized"
system, and to
            : manage and debug your containers.


8. What is wrong here? Anything wrong with this LXC version? Anything wrong
with the setup?

Thanks!

-- 
Angel Lopez
http://futur3.com/
... the geeks shall inherit the Earth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20180314/500bdf81/attachment.html>