[lxc-users] macvlan-based networking for unprivileged containers

[Serge Hallyn]

Quoting Fajar A. Nugraha (list at fajar.net):
> On Mon, Feb 16, 2015 at 9:52 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> > Quoting overlay fs (overlayfs at gmail.com):
> 
> >> > > However veth works
> >> > > just fine. And you don't have to put your public link (e.g. eth0) on
> >> > > bridge mode to have a working container with veth network.
> >> >
> >> > FWIW what it would take is an extension to lxc-user-nic to support
> >> > (accounted) unpriv macvlan.  /etc/lxc/lxc-usernet would then support
> >> > something like "$user macvlan eth0 10".
> >> >
> >> > But as Fajar says, the value of this seems dubious, and I'm not sure
> >> > whether that would have the same snooping-on-same-link concerns
> >> > that you'd have with a bridged eth0.
> >>
> >> Is there presently a way to block network traffic between unprivileged
> >> containers, or between a container and the host?  This could be
> >> desirable when running untrusted containers.
> >
> > You (your administrator) could create separate bridges for each user.
> 
> It might be useful to enhance lxc-user-nic to allow:
> - setting lxc.network.veth.pair

To do this, we'd have to provide a way for admins to specify which
pair names a given user may use.

But I see below you'd be doing that anyway,

> - allow veth without bridge (i.e. no lxc.network.link line on config file)
>
> With those two capabilities you could make routed setup without any
> bridge, where all containers route their traffic thru the host similar
> to the way pptp works. Containers can have IPs in the same segment as
> eth0, but can't see traffic meant to other IPs thru link-snooping. In
> this setup you DON'T need separate bridges for each user/container,
> but you DO need a config stanza (including fixed IP allocation) on
> host's /etc/network/interfaces for each container.
>
> This setup currently works on my test setup, privileged container. It
> also works for have root-started unprivileged container (i.e. created
> and started by root in /var/lib/lxc, but uses "lxc.include =
> /usr/share/lxc/config/ubuntu.userns.conf" and lxc.id_map) since it
> doesn't use lxc-user-nic. It does NOT work user-started unprivileged
> container.
> 
> Assuming:
> - your public link is eth0, 192.168.124.30/24 (LAN address in my test setup)
> - your containers (c1 and c2) gets IP address 192.168.124.251 and
> 192.168.124.252
> - you allocate private IP 172.16.0.1 for container's gateway (can be
> any private IP of your choice)
> 
> 
> ##########
> Host setup
> ##########
> 
> /etc/network/interfaces (if using ubuntu).
> ###
> auto lo
> iface lo inet loopback
> 
> auto eth0
> iface eth0 inet static
>         address  192.168.124.130
>         netmask 255.255.255.0
>         gateway 192.168.124.1
> 
> # c1's veth name on host side
> auto v-c1-0
> iface v-c1-0 inet static

I'm probably just ignorant here, but - does this not cause 'ifup -a' to
fail when the containers are not up?

>         address 172.16.0.1/32
>         scope link
>         pointopoint 192.168.124.251
> 
> # c2's veth name on host side
> auto v-c2-0
> iface v-c2-0 inet static
>         # note that this is the same IP as above, not a typo
>         address 172.16.0.1/32
>         scope link
>         # c2's IP
>         pointopoint 192.168.124.252
> ###