[lxc-users] Networking issues with LXC containers in EC2

[Fajar A. Nugraha]

On Tue, Jan 12, 2016 at 9:29 PM, Peter Steele <pwsteele at gmail.com> wrote:
> On 01/12/2016 05:59 AM, Fajar A. Nugraha wrote:
>>
>> On Tue, Jan 12, 2016 at 8:40 PM, Peter Steele <pwsteele at gmail.com> wrote:
>>>
>>> I should have added that I have no issue running our software on a single
>>> EC2 instance with containers running on that instance. We can assign
>>> multiple IPs to the instance itself, as well as to the containers running
>>> under the instance, and the containers can all communicate with each
>>> other
>>> as well as with the host.
>>
>>
>> can the containers in that setup communicate with systems outside the
>> host (e.g. access the internet)?
>>
>> if "no", then you might hit the multiple mac problem
>
> Sadly the answer is no. They cannot even ping another host in the same
> VPC...

Looks like multiple mac problem. As in, EC2 only allows one mac from
your interface.

Proxyarp should work:

(1) Make SURE your EC2 instances (I'd call them "host" from now on)
supports multiple IPs (private or elastic/public IPs, depending on
your needs). The easiest way is to add those IPs to your host
interface, make sure that that new IP can be accessed (e.g. ping that
IP from another host), and then remove it.

(2) Enable proxy arp on the host

echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp

of course, adjust to your environment (e.g. change interface name if
needed). You can also add entries in /etc/sysctl.conf or /etc/sysctl.d
so that this setting will persist on reboot.

(3) See https://www.mail-archive.com/lxc-users@lists.linuxcontainers.org/msg02380.html

This should make all outgoing packets use eth0's MAC, and the host
will effectively function as a router.

-- 
Fajar