[lxc-users] Filtering container traffic with iptables on host

[Björn Fischer]

Hello LXC-Users,

here at the CeBiTec, Bielefeld University, I'm currently looking
into replacements for Solaris zones. Now that you can even use
Docker in unprivileged LXD containers, an LXD based solution
seems to  be a candidate for larger deployment.

But there still is one issue I have with LXD containers. I want
to be able to filter traffic from and to specific LXD containers
on the host  with iptables. I approached this using cgroups and
net_cls. While this works for processes on the host,  it does not
work for processes in containers.

Here is a transscript of what I tried so far:

------snip------
root at drax:/root# lxc --version
2.17
root at drax:/root# lxc profile show default
config:
  environment.http_proxy: ""
  user.network_mode: ""
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: macvlan
    parent: eth0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: default
used_by: []
root at drax:/root# lxc launch ubuntu-daily:16.04 moonraker -p default
Creating moonraker
Starting moonraker
root at drax:/root# mkdir /sys/fs/cgroup/net_cls/ping
root at drax:/root# echo 1234 > /sys/fs/cgroup/net_cls/ping/net_cls.classid
[ starting ping on host ]
root at drax:/root# ps -ef | grep ping
root      3897  3187  0 13:23 pts/3    00:00:00 ping 10.0.160.1
root at drax:/root# echo 3897 > /sys/fs/cgroup/net_cls/ping/tasks
root at drax:/root# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
root at drax:/root# iptables -A OUTPUT -m cgroup --cgroup 1234 -j DROP
[ ping fails ]
root at drax:/root# iptables -D OUTPUT -m cgroup --cgroup 1234 -j DROP
[ ping operational again, iptables working as advertised ]

[ now start ping in container ]
root at drax:/root# lxc shell moonraker
run-parts: /etc/update-motd.d/98-fsck exited with return code 1

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

mesg: ttyname failed: Success
root at moonraker:~# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
	address 10.0.160.33
	netmask 255.255.255.0
	network	10.0.160.0
	gateway	10.0.160.1
	broadcast 10.0.160.255
root at moonraker:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:16:3e:d0:ae:30
          inet addr:10.0.160.33 Bcast:10.0.160.255 Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fed0:ae30/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1066 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:69641 (69.6 KB)  TX bytes:9982 (9.9 KB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:20 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1572 (1.5 KB)  TX bytes:1572 (1.5 KB)

root at moonraker:~# ping 10.0.160.1
PING 10.0.160.1 (10.0.160.1) 56(84) bytes of data.
64 bytes from 10.0.160.1: icmp_seq=1 ttl=255 time=5.04 ms
64 bytes from 10.0.160.1: icmp_seq=2 ttl=255 time=0.598 ms
[...]

[ on host try iptables again ]
root at drax:/root#
root at drax:/root# ps -ef | grep ping
558752    4684  4029  0 13:31 pts/6    00:00:00 ping 10.0.160.1
root at drax:/root# echo 4684 > /sys/fs/cgroup/net_cls/ping/tasks
root at drax:/root# iptables -A OUTPUT -m cgroup --cgroup 1234 -j DROP
[ ping continues ]
root at drax:/root# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere      cgroup 1234
[ iptables rule is active but has no effect on ping in container ]
------snip------

Certainly I am missing something very obvious.
If anyone could point me in the right direction,
I would appreciate that.

Thanks for your help.

Björn Fischer