<div dir="ltr">Hey Benoit and Fajar,<div><br></div><div>Thanks a lot for sharing those solutions. I will try those out soon and will let you know if I need help with anything.</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Oct 20, 2015 at 10:25 AM Benoit GEORGELIN - Association Web4all <<a href="mailto:benoit.georgelin@web4all.fr">benoit.georgelin@web4all.fr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-family:arial,helvetica,sans-serif;font-size:10pt;color:#000000"><div>I'm using an openvswith to provide network isolation/rules.<br></div><div><br></div><div>- Containers cannot change their own IP or MAC address. They will not receive any traffic anymore. </div><div>- I'm also able to manage network queue and limit traffic </div><div><br></div><div>Here is what I shared some time ago on that talk ( <a href="https://www.mail-archive.com/lxc-users@lists.linuxcontainers.org/msg03609.html" target="_blank">https://www.mail-archive.com/lxc-users@lists.linuxcontainers.org/msg03609.html</a> ) </div><div><br></div><div><span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none;background-color:#ffffff">-------------------------------------------</span></div><div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff">This is finally what I did with openvswitch : </div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff">ovs-ofctl del-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span>s vswitch-vps<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_GW ip actions=NORMAL"<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_GW arp actions=NORMAL"<br><br># default drop communication with HOST_A<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_HOST_A priority=38000 idle_timeout=0 action=drop"<br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff"># default drop communication with HOST_B</span><br style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_HOST_B priority=38000 idle_timeout=0 action=drop"</span></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br># Allow GW communication + Hypervisor<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_GW priority=39000 dl_type=0x0800 nw_src=IP_GW dl_src=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">MAC_GW</span> idle_timeout=0 action=normal"<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_GW priority=38500 dl_type=0x0806 dl_src=MAC_GW idle_timeout=0 action=normal"<br><br># Allow HOST A<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=PORT_HOST_A priority=38400 dl_type=0x0800 nw_src=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">IP_HOST_A</span> dl_src=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">MAC_HOST_A</span> idle_timeout=0 action=normal"<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">PORT_HOST_A</span> priority=38300 dl_type=0x0806 dl_src=MAC_HOST_A idle_timeout=0 action=normal"</div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"># Allow HOST B<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">PORT_HOST_B</span> priority=38400 dl_type=0x0800 nw_src=IP_HOST_B dl_src=MAC_HOST_B idle_timeout=0 action=normal"<br>ovs-ofctl add-<span style="background-color:#fffec4"><span style="background-color:#fffec4">flow</span></span> vswitch-vps "in_port=<span style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important;background-color:#ffffff">PORT_HOST_A</span> priority=38300 dl_type=0x0806 dl_src=MAC_HOST_B idle_timeout=0 action=normal"<br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff">To find port numbers: </div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff"><br></div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff">ovs-ofctl show BRIDGE</div><div style="color:#000000;font-family:arial,helvetica,sans-serif;font-size:13.3333330154419px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:#ffffff">-------------------------------------------</div></div><div><br></div><div>I did a small video about how I can deploy an LXC container (unprivileged) including network configuration.  </div><div><br></div><div><a href="https://vimeo.com/142828076" target="_blank">https://vimeo.com/142828076</a></div><div><br></div><div><br></div><div>I'm working to make a better integrated solution and I'll share that on github. </div><div><br></div><div>Cheers,</div><div><br></div><div><br></div><div><div><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif">Cordialement,</span><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold"><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold"><br></span></span></div><div><br></div><div><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold">Benoît Georgelin </span><br style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold"><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold"><br></span><span style="color:#c0c0c0;font-weight:bold;font-size:xx-small" size="1"><span style="font-family:times new roman,new york,times,serif;font-style:italic">Afin de contribuer au respect de l'environnement, merci de n'imprimer ce mail qu'en cas de nécessité</span></span><span style="color:rgb(51,51,51);font-family:times new roman,new york,times,serif;font-weight:bold"><br></span></div></div><br><hr><div><b>De: </b>"Fajar A. Nugraha" <<a href="mailto:list@fajar.net" target="_blank">list@fajar.net</a>><br><b>À: </b>"lxc-users" <<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a>><br><b>Envoyé: </b>Mardi 20 Octobre 2015 07:22:57<br><b>Objet: </b>Re: [lxc-users] Network isolation in unprivileged containers<br></div></div></div><div><div style="font-family:arial,helvetica,sans-serif;font-size:10pt;color:#000000"><br><div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Oct 20, 2015 at 6:11 PM, Akshay Karle <span dir="ltr"><<a href="mailto:akshay.a.karle@gmail.com" target="_blank">akshay.a.karle@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It would help to know, what level of isolation you're thinking about?<br>
What is the final end goal?<br></blockquote><br></span><div>I'm currently looking at ways to prevent any container from having the ability to discover other containers in the network and sniff their packets sent, which if sent over an unencrypted protocol (http for example) might be harmful as it could expose data.</div><br></div></div></blockquote><br><div>"Discover" and "sniff other container's packets" are two different things.</div><br><div>For example, on a routed setup where each container gets a /32 address, they can still ping each other (thus discovering the others exist), but they can't sniff traffic other than their own</div><br><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div>I'm now considering setting up iptable rules on the host to achieve this but don't have much experience with iptables so will do my research now to see what is needed to setup the right iptable rules. </div></div></div>
<br></blockquote><br><div>You mentioned you tried creating bridges for each container?</div><br><div>Combine that with direct /32 routing and proxyarp, and you pretty much confine each container to their own /32 address space. They will not be able to sniff other containers traffic. They won't even be able to use another IP address other than the one assigned to them.</div><br><div>I believe there was also similar-resultng technique with openvswitch(?) discussed some time ago on this list. Perhaps you can find it on the list archives, I don't have the link handy right now.</div><br><div>-- </div><div>Fajar</div></div></div></div>
<br>_______________________________________________<br>lxc-users mailing list<br><a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br><a href="http://lists.linuxcontainers.org/listinfo/lxc-users" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a><br></div></div></div>_______________________________________________<br>
lxc-users mailing list<br>
<a href="mailto:lxc-users@lists.linuxcontainers.org" target="_blank">lxc-users@lists.linuxcontainers.org</a><br>
<a href="http://lists.linuxcontainers.org/listinfo/lxc-users" rel="noreferrer" target="_blank">http://lists.linuxcontainers.org/listinfo/lxc-users</a></blockquote></div>