<div dir="ltr">Hi,<br><br><span style="font-size:10pt;font-family:sans-serif">I need to limit the
network bandwidth available to each LXC container using cgroup's net_cls.classid
feature. Each LXC container would have its own</span>
<br><span style="font-size:10pt;font-family:sans-serif">classid value
in such a way that all packets from containers would be tagged with the
classid and afterwards classified in the correct host configured traffic
class where the bandwidht limit applies. </span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">To achieve this,
I followed these steps:</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">1. Configure traffic
control:</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># tc qdisc del
dev eno54 root</span>
<br><span style="font-size:10pt;font-family:sans-serif"># tc qdisc add
dev eno54 root handle 10: htb</span>
<br><span style="font-size:10pt;font-family:sans-serif"># tc class add
dev eno54 parent 10: classid 10:1 htb rate 10mbit</span>
<br><span style="font-size:10pt;font-family:sans-serif"># tc class add
dev eno54 parent 10: classid 10:2 htb rate 50mbit</span>
<br><span style="font-size:10pt;font-family:sans-serif"># tc filter add
dev eno54 parent 10: protocol ip handle 1: cgroup</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">The device eno54
is the physical network interface that connect the host with the network.
It's part of the bridge where container virtual network interfaces are
added.</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># brctl show br0</span>
<br><span style="font-size:10pt;font-family:sans-serif">bridge name
bridge id STP enabled
interfaces</span>
<br><span style="font-size:10pt;font-family:sans-serif">br0
8000.00163ee2fda2 no
eno54</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">2. Set the classid
value in container config file.</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">lxctest1 container
config file has: lxc.cgroup.net_cls.classid = 0x00100001</span>
<br><span style="font-size:10pt;font-family:sans-serif">lxctest2 container
config file has: lxc.cgroup.net_cls.classid = 0x00100002</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">3. Start both
containers. Check that classid is correct and that they belong to the bridge.</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># lxc-start -n
lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif"># lxc-start -n
lxctest2</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># cat /sys/fs/cgroup/net_cls/lxc/<wbr>lxctest1/net_cls.classid</span>
<br><span style="font-size:10pt;font-family:sans-serif">1048577</span>
<br><span style="font-size:10pt;font-family:sans-serif"># cat /sys/fs/cgroup/net_cls/lxc/<wbr>lxctest2/net_cls.classid</span>
<br><span style="font-size:10pt;font-family:sans-serif">1048578</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># brctl show br0</span>
<br><span style="font-size:10pt;font-family:sans-serif">bridge name
bridge id STP enabled
interfaces</span>
<br><span style="font-size:10pt;font-family:sans-serif">br0
8000.00163ee2fda2 no
eno54</span>
<br><span style="font-size:10pt;font-family:sans-serif">
veth0-lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">
veth0-lxctest2</span>
<br><span style="font-size:10pt;font-family:sans-serif">4. Start iperf
in both containers. </span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">Expected behaviour:
iperf running on container lxctest1 being limited to 10 Mbps and iperf
running on lxctest2 container being limited to 50 Mbps.</span>
<br><span style="font-size:10pt;font-family:sans-serif">What I get: both
iperf running unconstrained at maximum speed.</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">5. I took the
iperf process running on lxctest1 container and checked that it was in
the tasks of the cgroup</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># pstree -c -p
37108</span>
<br><span style="font-size:10pt;font-family:sans-serif">lxc-start(37108)───systemd(<wbr>37118)─┬─agetty(37167)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
├─agetty(37168)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
├─dbus-daemon(37157)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
├─rsyslogd(37156)─┬─{rsyslogd}<wbr>(37161)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
│
└─{rsyslogd}(37162)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
├─sshd(37336)───sshd(41156)───<wbr>bash(41167)───iperf3(41523)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
├─systemd-journal(37131)</span>
<br><span style="font-size:10pt;font-family:sans-serif">
└─systemd-logind(37153)</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># cat /sys/fs/cgroup/net_cls/lxc/<wbr>lxctest1/tasks</span>
<br><span style="font-size:10pt;font-family:sans-serif">37118</span>
<br><span style="font-size:10pt;font-family:sans-serif">37131</span>
<br><span style="font-size:10pt;font-family:sans-serif">37153</span>
<br><span style="font-size:10pt;font-family:sans-serif">37156</span>
<br><span style="font-size:10pt;font-family:sans-serif">37157</span>
<br><span style="font-size:10pt;font-family:sans-serif">37161</span>
<br><span style="font-size:10pt;font-family:sans-serif">37162</span>
<br><span style="font-size:10pt;font-family:sans-serif">37167</span>
<br><span style="font-size:10pt;font-family:sans-serif">37168</span>
<br><span style="font-size:10pt;font-family:sans-serif">37336</span>
<br><span style="font-size:10pt;font-family:sans-serif">39618</span>
<br><span style="font-size:10pt;font-family:sans-serif">41156</span>
<br><span style="font-size:10pt;font-family:sans-serif">41167</span>
<br><span style="font-size:10pt;font-family:sans-serif">41523</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># cat /proc/41523/cgroup</span>
<br><span style="font-size:10pt;font-family:sans-serif">10:memory:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">9:hugetlb:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">8:perf_event:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">7:cpuset:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">6:devices:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">5:net_cls,net_prio:/lxc/<wbr>lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">4:blkio:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">3:cpu,cpuacct:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">2:freezer:/lxc/lxctest1</span>
<br><span style="font-size:10pt;font-family:sans-serif">1:name=systemd:/user.slice/<wbr>user-0.slice/session-1288.<wbr>scope/user.slice/user-0.slice/<wbr>session-1288.scope</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">6. I don't know
how to check that packets going out the container are actually being tagged
with the classid value, but the reality is that packets are not filtered
acording this value on the host and are not going to the correct class,
where bandwidth limit is applied.</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">7. I'm using Oracle
Linux 7 and the standard lxc package delivered in this distribution. Versions:</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># uname -a</span>
<br><span style="font-size:10pt;font-family:sans-serif">Linux exapru-aa.dit.aeat
4.1.12-112.14.15.el7uek.x86_64 #2 SMP Thu Feb 8 09:58:19 PST 2018 x86_64
x86_64 x86_64 GNU/Linux</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># cat /etc/oracle-release</span>
<br><span style="font-size:10pt;font-family:sans-serif">Oracle Linux Server
release 7.4</span>
<br>
<br><span style="font-size:10pt;font-family:sans-serif"># yum info lxc</span>
<br><span style="font-size:10pt;font-family:sans-serif">Loaded plugins:
ulninfo</span>
<br><span style="font-size:10pt;font-family:sans-serif">Installed Packages</span>
<br><span style="font-size:10pt;font-family:sans-serif">Name
: lxc</span>
<br><span style="font-size:10pt;font-family:sans-serif">Arch
: x86_64</span>
<br><span style="font-size:10pt;font-family:sans-serif">Version
: 1.1.5</span>
<br><span style="font-size:10pt;font-family:sans-serif">Release
: 2.0.9.el7</span>
<br><span style="font-size:10pt;font-family:sans-serif">Size
: 725 k</span>
<br><span style="font-size:10pt;font-family:sans-serif">Repo
: installed</span>
<br><span style="font-size:10pt;font-family:sans-serif">From repo
: ol7_latest</span>
<br><span style="font-size:10pt;font-family:sans-serif">Summary
: Linux Containers userspace tools</span>
<br><span style="font-size:10pt;font-family:sans-serif">URL
: </span><a href="http://linuxcontainers.org/" target="_blank"><span style="font-size:10pt;color:blue;font-family:sans-serif">http://linuxcontainers.org</span></a>
<br><span style="font-size:10pt;font-family:sans-serif">License
: LGPLv2+</span>
<br><span style="font-size:10pt;font-family:sans-serif">Description :
Containers are insulated areas inside a system, which have their own namespace</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: for filesystem, network, PID, IPC, CPU and
memory allocation and which can be</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: created using the Control Group and Namespace
features included in the Linux</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: kernel.</span>
<br><span style="font-size:10pt;font-family:sans-serif">
:</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: This package provides the lxc-* tools, which
can be used to start a single</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: daemon in a container, or to boot an entire
"containerized" system, and to</span>
<br><span style="font-size:10pt;font-family:sans-serif">
: manage and debug your containers.</span>
<br>
<br>
<br><span style="font-size:10pt;font-family:sans-serif">8. What is wrong
here? Anything wrong with this LXC version? Anything wrong with the setup?</span><br><br clear="all"><div><div>Thanks!<br><br></div><div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Angel Lopez<br><a href="http://futur3.com/" target="_blank">http://futur3.com/</a><br>... the geeks shall inherit the Earth</div>
</div></div></div>