[Lxc-users] Packet forwarding performance drop with 1000 containers

[Andrew Grigorev]

What gc_thresh* values did you set?.. Having gc_interval=30 should not 
be a bad thing if you have a proper gc_thresh1 value. If you would 
disable garbage collector, as you did by setting large gc_interval, then 
your system could be accidentally DoS'ed by stopping/starting your 
containers, for example.

25.04.2013 21:28, Benoit Lourdelet пишет:
> Hello,
>
> Working with 1000 containers I had already modified gc_thresh* to fit my
> needs.
>
> By mistake I had set gc_interval to a too high value (past 2^32) , forcing
> linux to set gc_interval to the default value (30) with is not suitable in
> my case.
>
> Setting gc_interval to 3600000 solved my problem.
>
> Thanks for pointing me in the right direction
>
> Benoit
>
> On 24/04/2013 07:21, "Guido Jäkel" <G.Jaekel at DNB.DE> wrote:
>
>> Dear Benoit,
>>
>> there's a lot of local matching and translation between layer2 and layer3
>> in your case. I wounder if it is related to the apr cache size and
>> garbage parameters. I found [http://linux.die.net/man/7/arp]:
>>
>>   gc_interval (since Linux 2.2)
>>      How frequently the garbage collector for neighbor entries should
>> attempt to run. Defaults to 30 seconds.
>>   gc_stale_time (since Linux 2.2)
>>      Determines how often to check for stale neighbor entries. When a
>> neighbor entry is considered stale, it is resolved again before sending
>> data to it. Defaults to 60 seconds.
>>   gc_thresh1 (since Linux 2.2)
>>      The minimum number of entries to keep in the ARP cache. The garbage
>> collector will not run if there are fewer than this number of entries in
>> the cache. Defaults to 128.
>>   gc_thresh2 (since Linux 2.2)
>>      The soft maximum number of entries to keep in the ARP cache. The
>> garbage collector will allow the number of entries to exceed this for 5
>> seconds before collection will be performed. Defaults to 512.
>>   gc_thresh3 (since Linux 2.2)
>>      The hard maximum number of entries to keep in the ARP cache. The
>> garbage collector will always run if there are more than this number of
>> entries in the cache. Defaults to 1024.
>>
>> This still seems to be the default on recent kernels, on a box with 3.3.5
>> I found
>>
>>    root at bladerunner9 ~ # cat /proc/sys/net/ipv4/neigh/default/gc*
>>    30
>>    60
>>    128
>>    512
>>    1024
>>
>> If the ARP cache get's exhausted, there must be continous additional ARP
>> resolution traffic and latency. May you check this theory?
>>
>>
>> Greetings
>>
>> Guido
>>
>>
>> On 2013-04-23 23:34, Benoit Lourdelet wrote:
>>> Hello,
>>>
>>> Forwarding throughput is decreasing gradually as I add containers. I
>>> don't
>>> see any sudden drop.
>>>
>>> I we consider aggregated forwarding performance with 100 containers to
>>> be
>>> 1, here are the measurements for
>>>
>>> # containers 	Aggregated throughput
>>> ------------------------------------
>>> 100 			1
>>> 500 			.71
>>> 1000 			.27
>>> 1100 			.23
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Lxc-users mailing list
> Lxc-users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lxc-users