<br><div class="gmail_quote">2009/8/24 KAMEZAWA Hiroyuki <span dir="ltr"><<a href="mailto:kamezawa.hiroyu@jp.fujitsu.com">kamezawa.hiroyu@jp.fujitsu.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On Sun, 23 Aug 2009 23:12:24 +0200<br>
Krzysztof Taraszka <<a href="mailto:krzysztof.taraszka@gnuhosting.net">krzysztof.taraszka@gnuhosting.net</a>> wrote:<br>
<br>
> :) excelent :)<br>
><br>
> my ugly patch printing right now undefinied data but the idea was the same<br>
> :)<br>
> how about memsw_limit for swap? :><br>
> I am looking for swap usage statistics from cgroup right now from<br>
> memcontrol.c :) but as you did the idea is good and should be add to the<br>
> kernel and lxc-tools :)<br>
><br>
<br>
</div>Hmm, why meminfo is necessary ? For cheating top/free/... etc ?<br>
<br>
</blockquote><div> <br>No, but if I am going to limit memory resources of container I would like to see inside this container memory limits when using userspace tools (such as top/free/...)<br><br>-- <br>Krzysztof Taraszka<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br><div><div class="h5">
><br>
> 2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr">daniel.lezcano@free.fr</a>><br>
><br>
> > Krzysztof Taraszka wrote:<br>
> ><br>
> >> 2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr">daniel.lezcano@free.fr</a>><br>
> >><br>
> >> Krzysztof Taraszka wrote:<br>
> >>><br>
> >>> 2009/8/23 Krzysztof Taraszka <<a href="mailto:krzysztof.taraszka@gnuhosting.net">krzysztof.taraszka@gnuhosting.net</a>><br>
> >>>><br>
> >>>><br>
> >>>><br>
> >>>> 2009/8/23 Krzysztof Taraszka <<a href="mailto:krzysztof.taraszka@gnuhosting.net">krzysztof.taraszka@gnuhosting.net</a>><br>
> >>>>><br>
> >>>>><br>
> >>>>><br>
> >>>>> 2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr">daniel.lezcano@free.fr</a>><br>
> >>>>>><br>
> >>>>>><br>
> >>>>>><br>
> >>>>>> Krzysztof Taraszka wrote:<br>
> >>>>>>><br>
> >>>>>>><br>
> >>>>>>><br>
> >>>>>>> 2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr">daniel.lezcano@free.fr</a>><br>
> >>>>>>>><br>
> >>>>>>>> Krzysztof Taraszka wrote:<br>
> >>>>>>>><br>
> >>>>>>>><br>
> >>>>>>>> Hello,<br>
> >>>>>>>>><br>
> >>>>>>>>><br>
> >>>>>>>>> I am running lxc on my debian unstable sandbox and I have a few<br>
> >>>>>>>>>> question<br>
> >>>>>>>>>> about memory managament inside linux containers based on lxc<br>
> >>>>>>>>>> project.<br>
> >>>>>>>>>><br>
> >>>>>>>>>> I have got linux kernel 2.6.30.5 with enabled :<br>
> >>>>>>>>>><br>
> >>>>>>>>>> +Resource counter<br>
> >>>>>>>>>> ++ Memory Resource Controller for Control Groups<br>
> >>>>>>>>>> +++ Memory Resource Controller Swap Extension(EXPERIMENTAL)<br>
> >>>>>>>>>><br>
> >>>>>>>>>> lxc-checkconfig pass all checks.<br>
> >>>>>>>>>><br>
> >>>>>>>>>> I read about cgroups memory managament<br>
> >>>>>>>>>> (Documentation/cgroups/memory.txt)<br>
> >>>>>>>>>> and I tried to pass those value to my debian sandbox.<br>
> >>>>>>>>>><br>
> >>>>>>>>>> And...<br>
> >>>>>>>>>> 'free -m' and 'top/htop' still show all available memory inside<br>
> >>>>>>>>>> container<br>
> >>>>>>>>>> (also If I set 32M for lxc.cgroup.memory.limit_in_bytes and<br>
> >>>>>>>>>> lxc.cgroup.memory.usage_in_bytes; and 64M for<br>
> >>>>>>>>>> lxc.cgroup.memory.memsw.usage_in_bytes and<br>
> >>>>>>>>>> lxc.cgroup.memory.memsw.limit_in_bytes free and top show all<br>
> >>>>>>>>>> resources).<br>
> >>>>>>>>>><br>
> >>>>>>>>>> What I did wrong? Does the container always show all available<br>
> >>>>>>>>>> memory<br>
> >>>>>>>>>> resources without cgroup limitations?<br>
> >>>>>>>>>><br>
> >>>>>>>>>> At the first glance I would say the configuration is correct.<br>
> >>>>>>>>>><br>
> >>>>>>>>>><br>
> >>>>>>>>>> But AFAIR, the memory cgroup is not isolated, if you specify 32MB<br>
> >>>>>>>>> you<br>
> >>>>>>>>> will<br>
> >>>>>>>>> see all the memory available on the system either if you are not<br>
> >>>>>>>>> allowed to<br>
> >>>>>>>>> use more than 32MB. If you create a program which allocates 64MB<br>
> >>>>>>>>> within<br>
> >>>>>>>>> a<br>
> >>>>>>>>> container configured with 32MB, and you "touch" the pages (may be<br>
> >>>>>>>>> that<br>
> >>>>>>>>> can<br>
> >>>>>>>>> be done with one mmap call with the MAP_POPULATE option), you<br>
> >>>>>>>>> should<br>
> >>>>>>>>> see the<br>
> >>>>>>>>> application swapping and the "memory.failcnt" increasing.<br>
> >>>>>>>>><br>
> >>>>>>>>> IMHO, showing all the memory available for the system instead of<br>
> >>>>>>>>> showing<br>
> >>>>>>>>> the allowed memory with the cgroup is weird but maybe there is a<br>
> >>>>>>>>> good<br>
> >>>>>>>>> reason<br>
> >>>>>>>>> to do that.<br>
> >>>>>>>>><br>
> >>>>>>>>><br>
> >>>>>>>>><br>
> >>>>>>>>><br>
> >>>>>>>>><br>
> >>>>>>>>> Thank you Daniel for your reply.<br>
> >>>>>>>> I think that LXC should isolate memory available for containers like<br>
> >>>>>>>> Vserver<br>
> >>>>>>>> or FreeVPS do (memory + swap) if .cgroup.memory.* and<br>
> >>>>>>>> lxc.cgroup.memory.memsw.* is set.<br>
> >>>>>>>> Is there any possibility to make a patch for linux kernel /<br>
> >>>>>>>> lxc-tools<br>
> >>>>>>>> to<br>
> >>>>>>>> show the limitations inside cointainers propertly? I think is a good<br>
> >>>>>>>> idea<br>
> >>>>>>>> and it should be apply as soon as possible.<br>
> >>>>>>>><br>
> >>>>>>>><br>
> >>>>>>>><br>
> >>>>>>>> Maybe a solution can be to add a new memory.meminfo file in the<br>
> >>>>>>> same<br>
> >>>>>>> format than /proc/meminfo, so it will be possible to mount --bind<br>
> >>>>>>> /cgroup/foo/memory.meminfo to /proc/meminfo for the container.<br>
> >>>>>>><br>
> >>>>>>><br>
> >>>>>>><br>
> >>>>>>> Yes, I thought the same. This should allow the user-space tools<br>
> >>>>>> based on<br>
> >>>>>> /proc/meminfo (such as comand line "free") show limited information :)<br>
> >>>>>><br>
> >>>>>><br>
> >>>>>><br>
> >>>>>> Hmmm... does the memory.stat is a good start point for make new one<br>
> >>>>> object<br>
> >>>>> memory.meminfo similar to /proc/meminfo? If so, I can play by my self<br>
> >>>>> with<br>
> >>>>> lxc-tools code.<br>
> >>>>><br>
> >>>>><br>
> >>>>><br>
> >>>>><br>
> >>>> Hmmm... Daniel, I have got a question (that do I thinking in the right<br>
> >>>> way).<br>
> >>>> here is an output from /proc/meminfo from openvz:<br>
> >>>><br>
> >>>><br>
> >>>> MemTotal: 262144 kB<br>
> >>>> MemFree: 232560 kB<br>
> >>>> Buffers: 0 kB<br>
> >>>> Cached: 0 kB<br>
> >>>> SwapCached: 0 kB<br>
> >>>> Active: 0 kB<br>
> >>>> Inactive: 0 kB<br>
> >>>> HighTotal: 0 kB<br>
> >>>> HighFree: 0 kB<br>
> >>>> LowTotal: 262144 kB<br>
> >>>> LowFree: 232560 kB<br>
> >>>> SwapTotal: 0 kB<br>
> >>>> SwapFree: 0 kB<br>
> >>>> Dirty: 0 kB<br>
> >>>> Writeback: 0 kB<br>
> >>>> AnonPages: 0 kB<br>
> >>>> Mapped: 0 kB<br>
> >>>> Slab: 0 kB<br>
> >>>> SReclaimable: 0 kB<br>
> >>>> SUnreclaim: 0 kB<br>
> >>>> PageTables: 0 kB<br>
> >>>> NFS_Unstable: 0 kB<br>
> >>>> Bounce: 0 kB<br>
> >>>> WritebackTmp: 0 kB<br>
> >>>> CommitLimit: 0 kB<br>
> >>>> Committed_AS: 0 kB<br>
> >>>> VmallocTotal: 0 kB<br>
> >>>> VmallocUsed: 0 kB<br>
> >>>> VmallocChunk: 0 kB<br>
> >>>> HugePages_Total: 0<br>
> >>>> HugePages_Free: 0<br>
> >>>> HugePages_Rsvd: 0<br>
> >>>> HugePages_Surp: 0<br>
> >>>> Hugepagesize: 2048 kB<br>
> >>>><br>
> >>>> most of values are 0.<br>
> >>>><br>
> >>>> I have an question about SwapTotal and SwapFree for LXC.<br>
> >>>> As I thinking that:<br>
> >>>><br>
> >>>> MemTotal might be: hierarchical_memory_limit<br>
> >>>> MemFree might be: hierarchical_memory_limit - cache<br>
> >>>><br>
> >>>><br>
> >>>> I am not a memory expert, but isn't MemFree : hierarchical_memory_limit<br>
> >>> -<br>
> >>> rss ?<br>
> >>><br>
> >>> the<br>
> >>>><br>
> >>>> SwapTotal might be: hierarchical_memsw_limit<br>
> >>>> SwapFree might be: hierarchical_memsw_limit - rss<br>
> >>>><br>
> >>>> rss - # of bytes of anonymous and swap cache memory<br>
> >>>> I don't know at all that hierarchical_memsw_limit is an good value for<br>
> >>>> swap<br>
> >>>> total, because as I read it is a mem+swap at all.<br>
> >>>><br>
> >>>> Does the lxc memory.meminfo might look like above? Where can I get the<br>
> >>>> Hugepagesize?<br>
> >>>><br>
> >>>><br>
> >>>> Right, I agree most of the interesting information to create a<br>
> >>> memory.meminfo is already there in another file and another format.<br>
> >>> Probably<br>
> >>> some informations in memory.stat can be moved to memory.meminfo and this<br>
> >>> one<br>
> >>> can be step by step filled with cgroup memory statistic informations.<br>
> >>> IMO,<br>
> >>> if the memory controller displays memory statistics like a proc/meminfo<br>
> >>> file<br>
> >>> format, that will make consistency for these informations and make<br>
> >>> trivial<br>
> >>> the isolation/virtualization with a simple mount-bind.<br>
> >>><br>
> >>><br>
> >>><br>
> >>> Hmm..<br>
> >> might be. Right now I am looking for and writing new function in<br>
> >> mm/memcontrol.c file for writing some stats in memory.meminfo file for<br>
> >> tests.<br>
> >> Dirty and ugly part of code, but if it will work as we thought<br>
> >> (mount-bind)<br>
> >> and as you wrote above, that will be very simple.<br>
> >> I am going to look how does the /proc/meminfo is doing by the openvz.<br>
> >> mm/memcontrol.c was wrote by xemul from openvz and balbir from ibm.<br>
> >> If I am thinking in the right way, guys from openvz made their own patch<br>
> >> for<br>
> >> meminfo based on the mm/memcontrol.c. If I am wrong - where do they taking<br>
> >> meminfo data? :)<br>
> >><br>
> ><br>
> > I did this ugly patch patch for MemTotal/MemFree - maybe wrong :)<br>
> ><br>
> > Index: linux-2.6/mm/memcontrol.c<br>
> > ===================================================================<br>
> > --- linux-2.6.orig/mm/memcontrol.c 2009-06-23 12:00:52.000000000 +0200<br>
> > +++ linux-2.6/mm/memcontrol.c 2009-08-23 22:49:02.000000000 +0200<br>
> > @@ -2200,6 +2200,27 @@ static int mem_cgroup_swappiness_write(s<br>
> > }<br>
> ><br>
> ><br>
> > +static int mem_cgroup_meminfo(struct cgroup *cgrp, struct cftype *cft,<br>
> > + struct seq_file *seq)<br>
> > +{<br>
> > +#define K(x) ((x) << 10)<br>
> > +<br>
> > + struct mem_cgroup *mem_cont = mem_cgroup_from_cont(cgrp);<br>
> > + struct mcs_total_stat mystat = { };<br>
> > + unsigned long long limit, memsw_limit;<br>
> > +<br>
> > + mem_cgroup_get_local_stat(mem_cont, &mystat);<br>
> > + memcg_get_hierarchical_limit(mem_cont, &limit, &memsw_limit);<br>
> > +<br>
> > + seq_printf(seq,<br>
> > + "MemTotal: %8llu kB\n"<br>
> > + "MemFree: %8llu kB\n",<br>
> > + limit / 1024, (limit - mystat.stat[MCS_RSS]) / 1024);<br>
> > +<br>
> > + return 0;<br>
> > +#undef K<br>
> > +}<br>
> > +<br>
> > static struct cftype mem_cgroup_files[] = {<br>
> > {<br>
> > .name = "usage_in_bytes",<br>
> > @@ -2242,6 +2263,10 @@ static struct cftype mem_cgroup_files[]<br>
> > .read_u64 = mem_cgroup_swappiness_read,<br>
> > .write_u64 = mem_cgroup_swappiness_write,<br>
> > },<br>
> > + {<br>
> > + .name = "meminfo",<br>
> > + .read_seq_string = mem_cgroup_meminfo,<br>
> > + },<br>
> > };<br>
> ><br>
> > #ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP<br>
> ><br>
> ><br>
> > With the lxc tools I did:<br>
> ><br>
> > lxc-execute -n foo /bin/bash<br>
> > echo 268435456 > /cgroup/foo/memory.limit_in_bytes<br>
> > mount --bind /cgroup/foo/memory.meminfo /proc/meminfo<br>
> > for i in $(seq 1 100); do sleep 3600 & done<br>
> ><br>
> > And the result for "free" is:<br>
> ><br>
> > free:<br>
> ><br>
> > total used free shared buffers cached<br>
> > Mem: 262144 9692 252452 0 0 0<br>
> > -/+ buffers/cache: 9692 252452<br>
> > Swap: 0 0 0<br>
> ><br>
> ><br>
> > and for "top":<br>
> ><br>
> > top - 22:57:37 up 8 min, 1 user, load average: 0.00, 0.02, 0.00<br>
> > Tasks: 104 total, 1 running, 103 sleeping, 0 stopped, 0 zombie<br>
> > Cpu(s): 0.3%us, 1.0%sy, 0.0%ni, 98.4%id, 0.0%wa, 0.0%hi, 0.3%si,<br>
> > 0.0%st<br>
> > Mem: 262144k total, 9864k used, 252280k free, 0k buffers<br>
> > Swap: 0k total, 0k used, 0k free, 0k cached<br>
> ><br>
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND<br>
> > 337 root 20 0 14748 1132 872 R 1.0 0.4 0:00.24 top<br>
> > 1 root 20 0 8136 484 408 S 0.0 0.2 0:00.00 lxc-init<br>
> > 2 root 20 0 89980 1724 1348 S 0.0 0.7 0:00.70 bash<br>
> > 25 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep<br>
> > 232 root 20 0 86916 616 524 S 0.0 0.2 0:00.00 sleep<br>
> > 233 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep<br>
> > 234 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep<br>
> > 235 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep<br>
> > .....<br>
> ><br>
> ><br>
> > :)<br>
> ><br>
> ><br>
</div></div>> _______________________________________________<br>
> Containers mailing list<br>
> <a href="mailto:Containers@lists.linux-foundation.org">Containers@lists.linux-foundation.org</a><br>
> <a href="https://lists.linux-foundation.org/mailman/listinfo/containers" target="_blank">https://lists.linux-foundation.org/mailman/listinfo/containers</a><br>
><br>
<br>
</blockquote></div><br>