[lxc-devel] NFS mounts inside a container uses/requires the host IP stack

[Tim Spriggs]

Hi All,

I have a host with a network setup that uses bonding/vlans/bridging
and the path to a typical container might look like:

bond0 -> vlan5 -> br5 -> veth-pair

Each vlan has it's own subnet (IPv4/IPv6 both deployed on most vlans).

I have a container (lxc1) with an IPv4 address on vlan5 and the host
itself is not supposed to have a vlan5 IPv4 address available. If I
try to mount an nfs filesystem inside lxc1 it will eventually fail
with a timeout even though nothing prevents communication between lxc1
and nfs1. I hit my head on this for some time before I realized what
might be going on; If I add an appropriate IPv4 address to br5 on the
host then the mount succeeds immediately.

Somewhere in the process of mounting an NFS filesystem the host
context is used to talk to the NFS server and so the virtualized eth
device can not be used for this communication and then the mount
fails.

Among other things this probably breaks host based access control from
the NFS server (nfs1 in this case)

I have the output of tshark for the session attached (unhealthy.dump)
and on line 43 you can see the HOST replying and taking over instead
of lxc1. I have also attached the syslog output related to rpcdebug
(unhealthy.rpcdebug) for when the host replies.

For comparison, I have also attached similar traces for when the host
has no IPv4 address on that network
(not-working.dump/not-working.rpcdebug). What happens is the host
stack decides that it is not directly attached to the vlan5 subnet and
so tries to use its default route on vlan3 to get to it. This ends up
not working at all since vlan3 can not talk to vlan5.

I am using the Debian kernel sources (linux-source-2.6.37/2.6.37-2)
with the addition of CONFIG_CGROUP_MEM_RES_CTLR and friends.

Is there any way to have full nfs isolation for a container?

Cheers,
-Tim

PS: While searching around I found another thread potentially related
to this issue. https://bugzilla.redhat.com/show_bug.cgi?id=500653
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unhealthy.dump
Type: application/octet-stream
Size: 7481 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20110315/ccfbaaf1/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: unhealthy.rpcdebug
Type: application/octet-stream
Size: 38888 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20110315/ccfbaaf1/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not-working.dump
Type: application/octet-stream
Size: 2476 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20110315/ccfbaaf1/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not-working.rpcdebug
Type: application/octet-stream
Size: 9637 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20110315/ccfbaaf1/attachment-0003.obj>