[Lxc-users] [MySQL] Weird performances problem between containers on a same host

[Daniel Lezcano]

On 08/11/2011 09:32 AM, Mauras Olivier wrote:
> So here's my results.
> On 55 packets transmitted for the mysql request
> 25 correct checksums
> 28 incorrect
>
> Disabling TSO and GSO doesn't help much, got exact same result

AFAIR, you have to disable all the offloading capabilities of the
interface before creating the container.

Anyway, it seems you hit this bug:

commit 79b569f0ec53a14c4d71e79d93a8676d9a0fda6d
Author: Daniel Lezcano <daniel.lezcano at free.fr>
Date:   Wed Mar 30 02:42:17 2011 -0700

    netdev: fix mtu check when TSO is enabled
   
    In case the device where is coming from the packet has TSO enabled,
    we should not check the mtu size value as this one could be bigger
    than the expected value.
   
    This is the case for the macvlan driver when the lower device has
    TSO enabled. The macvlan inherit this feature and forward the packets
    without fragmenting them. Then the packets go through dev_forward_skb
    and are dropped. This patch fix this by checking TSO is not enabled
    when we want to check the mtu size.
   
    Signed-off-by: Daniel Lezcano <daniel.lezcano at free.fr>
    Acked-by: "Eric W. Biederman" <ebiederm at xmission.com>
    Signed-off-by: David S. Miller <davem at davemloft.net>

IMO, if you have a redhat, you should ask them to incorporate this patch
in their kernel.

> As for the macvlan bridge mode how can i check if it's available or not??

Hard to say as the Redhat kernel has a lot of backport, not necessarly
identified. But at the first glance I would say it is supported
otherwise you would not be able to communicate.

> Would it let things work even if not supported?

No, if the bridge mode is not supported the containers won't be able to
communicate through the macvlan.

> --- Misc ---
> Veth pair device: enabled
> Macvlan: enabled
> Vlan: enabled
> File capabilities: enabled
>
>
> Thanks,
> Olivier
>
>
> On Wed, Aug 10, 2011 at 6:25 PM, Daniel Lezcano <daniel.lezcano at free.fr>wrote:
>
>> On 08/10/2011 05:54 PM, Daniel Lezcano wrote:
>>> On 08/10/2011 04:51 PM, Mauras Olivier wrote:
>>>> Hello,
>>>>
>>>> I have several containers running on a host - ~10
>>>> One of them is running a MySQL database. Several of the others are
>> running
>>>> php code under apache that fetch datas from the database.
>>>>
>>>> Host is using eth0, while my containers are on a bridge using eth1, and
>>>> configured in macvlan bridge mode. Host runs SL6 with 2.6.32 RH kernel -
>>>> Host is a VMWare ESX virtual machine for that matter.
>>>> Ping latency between containers is at an average of 0.050 ms
>>>>
>>>> What i noted, is that one webapp were getting very slow... After
>>>> investigating, the only thing that i could find, is that requests from
>>>> containers are _slower_ than from any other hosts.
>>>>
>>>> See below:
>>>>
>>>> container1 ~ # time (echo "select * from testsuites;" | mysql -h
>>>> container_mysql -u nmp -pxxx testlink)
>>>> id      details
>>>> 42
>>>> (... cut only 25 entries anyway)
>>>>
>>>> real    *0m0.875s*
>>>>
>>>> Time varies between 0.8 and 1.2s
>>>>
>>>> >From the host or another VM on the same network with the exact same
>> request:
>>>> real    *0m0.022s*
>>>>
>>>>
>>>> So that particular app that can loop over 19 requests takes sometimes up
>> to
>>>> 20s to display a page instead of ~0.5s from another host...
>>>>
>>>> Is there some sysctl/settings to tweak or it's just not relevant to make
>>>> requests between containers on the same host??
>>> Hmm, thanks for the detailed explanation.
>>>
>>> Can you check with tcpdump if there are problems with the patch checksums
>> ?
>>> And try to disable the TSO and SGO of eth1 if there are available ?
>> Oh, and another question. AFAIK, the bridge mode is available since the
>> 2.6.33 kernel.
>> If we try to enable the bridge mode on a macvlan while this is not
>> supported, no error is reported.
>> So I don't know if the RH kernel did backport the bridge mode in their
>> .32 kernel.
>>