[lxc-users] lxc-security: iptables audit with nflog not working with default settings (insecure)

Wed Mar 11 12:22:42 UTC 2015

On Wed, Mar 11, 2015 at 7:02 PM, Fiedler Roman <Roman.Fiedler at ait.ac.at> wrote:
> This should be exactly the configuration I have tested  so far. But that did
> not yet solve my problem ...
>
> * If some process in guest registers for the same NFLOG queue, he can "steal"
> the messages from the host queue, thus removing traces of his activity from
> host logging. SECURITY-ASPECT: apart from log corruption, the guest can get
> knowledge about any other connection to/from other containers and the host and
> as they include  sequence numbers, may be able to inject spoofed data into any
> other unencrypted TCP connection or at least interrupt the connection using
> another helper machine.

No. What makes you believe that?

Host and containers does not share iptables rules. Their entire
network stack is separated thru network namespace. There's no such
thing as "stealing the message".

A test would probably prove my statement faster. Try this on your
container, while keeping the same rules on the host side:

iptables -I INPUT 1 -d 192.168.124.173 -j NFLOG --nflog-group 0
--nflog-prefix lxc-v
iptables -I OUTPUT 1 -s 192.168.124.173 -j NFLOG --nflog-group 0
--nflog-prefix lxc-v

Note that on the container the chain has to be INPUT on OUTPUT instead
of FORWARD. Then test it. The container will logs this

Mar 11 12:10:56 v lxc-v IN= OUT=eth0 MAC= SRC=192.168.124.173
DST=192.168.124.134 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=26016 DF
PROTO=ICMP TYPE=8 CODE=0 ID=2087 SEQ=1 UID=0 GID=0 MARK=0
Mar 11 12:10:55 v lxc-v IN=eth0 OUT=
MAC=00:16:3e:f5:cd:94:00:16:3e:2e:d2:6d:08:00 SRC=192.168.124.134
DST=192.168.124.173 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=29087 PROTO=ICMP
TYPE=0 CODE=0 ID=2087 SEQ=1 MARK=0
Mar 11 12:10:57 v lxc-v IN= OUT=eth0 MAC= SRC=192.168.124.173
DST=192.168.124.134 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=26020 DF
PROTO=ICMP TYPE=8 CODE=0 ID=2087 SEQ=2 UID=0 GID=0 MARK=0
Mar 11 12:10:56 v lxc-v IN=eth0 OUT=
MAC=00:16:3e:f5:cd:94:00:16:3e:2e:d2:6d:08:00 SRC=192.168.124.134
DST=192.168.124.173 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=29287 PROTO=ICMP
TYPE=0 CODE=0 ID=2087 SEQ=2 MARK=0

... while at the same time, the host also logs it
Mar 11 19:10:55 utopic lxc-v IN=br0 OUT=br0
MAC=00:16:3e:2e:d2:6d:00:16:3e:f5:cd:94:08:00 SRC=192.168.124.173
DST=192.168.124.134 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=26016 DF
PROTO=ICMP TYPE=8 CODE=0 ID=2087 SEQ=1 MARK=0
Mar 11 19:10:55 utopic lxc-v IN=br0 OUT=br0
MAC=00:16:3e:f5:cd:94:00:16:3e:2e:d2:6d:08:00 SRC=192.168.124.134
DST=192.168.124.173 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=29087 PROTO=ICMP
TYPE=0 CODE=0 ID=2087 SEQ=1 MARK=0
Mar 11 19:10:56 utopic lxc-v IN=br0 OUT=br0
MAC=00:16:3e:2e:d2:6d:00:16:3e:f5:cd:94:08:00 SRC=192.168.124.173
DST=192.168.124.134 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=26020 DF
PROTO=ICMP TYPE=8 CODE=0 ID=2087 SEQ=2 MARK=0
Mar 11 19:10:56 utopic lxc-v IN=br0 OUT=br0
MAC=00:16:3e:f5:cd:94:00:16:3e:2e:d2:6d:08:00 SRC=192.168.124.134
DST=192.168.124.173 LEN=84 TOS=00 PREC=0x00 TTL=64 ID=29287 PROTO=ICMP
TYPE=0 CODE=0 ID=2087 SEQ=2 MARK=0

(note that my host and container uses different timezone, so the times
look 7-hours apart)

>
> * If a guest should be protected with iptables also, e.g. to avoid Apache or
> Tomcat to connect to the local SSH port, those error logs - which are useful
> to detect ongoing guest intrusions - do not make it to any log-file.

No.

Whatever the container does inside it does not matter to the host. The
host simply capture the packets passing thru the bridge (with
bridge-nf-call-iptables on), and doesn't really care what the
container does with it (block it, drop it, accept it, whatever).

On the host side, nflog does NOT stop iptables processing, so the
chains further down (e.g. the ones doing ACCEPT, or REJECT if you
want) will still see the packets. You can verify it with "iptables -nL
-v", and see that the hit counters for both the nflog rule AND
ACCEPT/REJECT rule increase.

>
> [...snip...]
>
>> You might only be missing the "bridge-nf-call-iptables" part. Note
>> that you shouldn't need it IF you use a custom lxc network setup which
>> doesn't use bridges:
>> https://www.mail-archive.com/lxc-users@lists.linuxcontainers.org/msg02587.html
>
> I've tried various network configurations also. I fear that effort here is
> quite futile since I do not yet understand the core kernel namespace concepts,

That will be your biggest problem.

Short version: when using containers installed using templates
(preferably download template), you need to treat container's
networking stack as separate (e.g. on a different "server") with the
one on the host.

The only "link" the host has to the container is the veth pair
(vethXXXX by default on host side, usually eth0 on the container
side). Think of lxcbr0 (or whatever bridge you use) as a "switch",
with vethXXXX as the switch port that is connected to the container.

> e.g. how nflink and user namespaces work together. Hence everything I could do
> is (inefficient) trial and error instead of controlled engineering. And in the
> end I cannot be sure that there are not reliability/security-relevant holes
> left open with trial/error.
>
> Without any other clues from the mailing list, I'll still try out this
> procedure also and see if it would change the nflog behavior.

Then spend some time to learn.

Googling "network namespace" is a good start. The lwn article should
be very helpful.

-- 
Fajar