[cgmanager-devel] read overhead

[Serge Hallyn]

Hi,

Google has said that they need to see 1000 reads/second from cgmanager.
The code's not really in a state yet where it should be measured this
way but I figured I'd get a baseline.

On my laptop (3yo 8-core i7), simply reading memory.limit_in_bytes from
bash 1000 times per second is not possible :)  A little under 2 seconds
is.  Here are the three scripts I tried.  The first simply reads
the cgroup file 1000 times.  The second uses dbus-send to send 1000
requests to the cgroup manager.  I would assume that using a custom
client which uses the cgmanager-client.c (which I assume will become a
library?) would turn out faster.  The third is there for a fairer
comparison to anything uses a daemon - it simply determins your
current cgroup and prints out the memory.usage_in_bytes there.  Any
daemon which is answering a generic request will have to do something
like that, of course.

Now google have said they would simply read from the cgroup fs, and
hoped to go through cgmanager for the writes - that of course is fine.
There might be another way to achieve (on faster machines than my
laptop :) 1000 r/s: we could have the cgmanager keep a short list of
open files for certain requestors, so we could avoid some of the
processing we are doing on every read.  (Just handing an open fd to
the caller probably is not ok, since the caller can reopen the file
read-write if the file is owned by them.  So to do that we'd have to
assume that the caller is 100% trusted, which would be a very special
case, and we may as well, at that point, let them mount the cgroupfs
rw).

Anyway, here are the 3 trivial scripts followed by the result of running
'time <script>'

xxxxxxxxxxx  timeme1.sh xxxxxxxxxxx
#!/bin/bash
for i in `seq 1 1000`; do
	cat /sys/fs/cgroup/memory/memory.usage_in_bytes
done

run 1:
real	0m1.975s
user	0m0.795s
sys	0m1.243s

xxxxxxxxxxx  timeme2.sh xxxxxxxxxxx
#!/bin/bash
for i in `seq 1 1000`; do
	dbus-send --print-reply --address=unix:path=/tmp/cgmanager --type=method_call /org/linuxcontainers/cgmanager org.linuxcontainers.cgmanager0_0.getValue string:'memory' string:'' string:'memory.usage_in_bytes'
done

run 1:
real	0m3.592s
user	0m1.394s
sys	0m1.731s

run 2:
real	0m3.587s
user	0m1.398s
sys	0m1.734s

xxxxxxxxxxx  timeme3.sh xxxxxxxxxxx
#!/bin/bash
for i in `seq 1 1000`; do
	cg=`awk -F: '/memory/ { print $3 }' /proc/self/cgroup`
	cat /sys/fs/cgroup/memory/$cg/memory.usage_in_bytes
done

run 1:
real    0m5.822s
user    0m2.685s
sys     0m3.333s
run 2:
real    0m5.826s
user    0m2.690s
sys     0m3.344s