[cgmanager-devel] read overhead

[Stéphane Graber]

On Mon, Dec 09, 2013 at 06:06:49PM -0500, Stéphane Graber wrote:
> On Mon, Dec 09, 2013 at 04:56:30PM -0600, Serge Hallyn wrote:
> > Hi,
> > 
> > Google has said that they need to see 1000 reads/second from cgmanager.
> > The code's not really in a state yet where it should be measured this
> > way but I figured I'd get a baseline.
> > 
> > On my laptop (3yo 8-core i7), simply reading memory.limit_in_bytes from
> > bash 1000 times per second is not possible :)  A little under 2 seconds
> > is.  Here are the three scripts I tried.  The first simply reads
> > the cgroup file 1000 times.  The second uses dbus-send to send 1000
> > requests to the cgroup manager.  I would assume that using a custom
> > client which uses the cgmanager-client.c (which I assume will become a
> > library?) would turn out faster.  The third is there for a fairer
> > comparison to anything uses a daemon - it simply determins your
> > current cgroup and prints out the memory.usage_in_bytes there.  Any
> > daemon which is answering a generic request will have to do something
> > like that, of course.
> > 
> > Now google have said they would simply read from the cgroup fs, and
> > hoped to go through cgmanager for the writes - that of course is fine.
> > There might be another way to achieve (on faster machines than my
> > laptop :) 1000 r/s: we could have the cgmanager keep a short list of
> > open files for certain requestors, so we could avoid some of the
> > processing we are doing on every read.  (Just handing an open fd to
> > the caller probably is not ok, since the caller can reopen the file
> > read-write if the file is owned by them.  So to do that we'd have to
> > assume that the caller is 100% trusted, which would be a very special
> > case, and we may as well, at that point, let them mount the cgroupfs
> > rw).
> > 
> > Anyway, here are the 3 trivial scripts followed by the result of running
> > 'time <script>'
> 
> For a more realistic measurement of the processing time, can you try the
> following?
> 
> ---
> #!/usr/bin/python3
> import dbus
> import time
> 
> bus = dbus.connection.Connection("unix:path=/tmp/cgmanager")
> 
> cgmanager = bus.get_object(None, '/org/linuxcontainers/cgmanager')
> start_time = time.time()
> for i in range(1000):
>     print(cgmanager.getValue("memory", "", "memory.usage_in_bytes"))
> end_time = time.time()
> print("Total time: %s" % (end_time - start_time))
> ---

Actually, you may want to just call getValue and not print the result if
you want the smallest possible overhead as it may impact the result a
tiny bit.

> 
> > 
> > xxxxxxxxxxx  timeme1.sh xxxxxxxxxxx
> > #!/bin/bash
> > for i in `seq 1 1000`; do
> > 	cat /sys/fs/cgroup/memory/memory.usage_in_bytes
> > done
> > 
> > run 1:
> > real	0m1.975s
> > user	0m0.795s
> > sys	0m1.243s
> > 
> > xxxxxxxxxxx  timeme2.sh xxxxxxxxxxx
> > #!/bin/bash
> > for i in `seq 1 1000`; do
> > 	dbus-send --print-reply --address=unix:path=/tmp/cgmanager --type=method_call /org/linuxcontainers/cgmanager org.linuxcontainers.cgmanager0_0.getValue string:'memory' string:'' string:'memory.usage_in_bytes'
> > done
> > 
> > run 1:
> > real	0m3.592s
> > user	0m1.394s
> > sys	0m1.731s
> > 
> > run 2:
> > real	0m3.587s
> > user	0m1.398s
> > sys	0m1.734s
> > 
> > xxxxxxxxxxx  timeme3.sh xxxxxxxxxxx
> > #!/bin/bash
> > for i in `seq 1 1000`; do
> > 	cg=`awk -F: '/memory/ { print $3 }' /proc/self/cgroup`
> > 	cat /sys/fs/cgroup/memory/$cg/memory.usage_in_bytes
> > done
> > 
> > run 1:
> > real    0m5.822s
> > user    0m2.685s
> > sys     0m3.333s
> > run 2:
> > real    0m5.826s
> > user    0m2.690s
> > sys     0m3.344s
> > _______________________________________________
> > cgmanager-devel mailing list
> > cgmanager-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/cgmanager-devel
> 
> -- 
> Stéphane Graber
> Ubuntu developer
> http://www.ubuntu.com



> _______________________________________________
> cgmanager-devel mailing list
> cgmanager-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/cgmanager-devel


-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/cgmanager-devel/attachments/20131209/997f55eb/attachment.pgp>