I added the swap resource monitoring to your patch. <br>I did it in the very simple way, because if: memsw = swap + mem, then swap_in_use == memsw.usage_in_bytes - memory.usage_in_bytes<br><br>simple patch:<br><br>+static int mem_cgroup_meminfo(struct cgroup *cgrp, struct cftype *cft,<br>
+ struct seq_file *seq)<br>+{<br>+#define K(x) ((x) << 10)<br>+<br>+ struct mem_cgroup *mem_cont = mem_cgroup_from_cont(cgrp);<br>+ struct mcs_total_stat mystat = { };<br>+ unsigned long long limit, memsw_limit;<br>
+<br>+ u64 swap_in_u, m_usage, s_usage;<br>+<br>+ s_usage = res_counter_read_u64(&mem_cont->memsw, RES_USAGE);<br>+ m_usage = res_counter_read_u64(&mem_cont->res, RES_USAGE);<br>+<br>+ swap_in_u = s_usage - m_usage;<br>
+<br>+ mem_cgroup_get_local_stat(mem_cont, &mystat);<br>+ memcg_get_hierarchical_limit(mem_cont, &limit, &memsw_limit);<br>+<br>+ seq_printf(seq,<br>+ "MemTotal: %8llu kB\n"<br>
+ "MemFree: %8llu kB\n"<br>+ "SwapTotal: %8llu kB\n"<br>+ "SwapFree: %8llu kB\n",<br>+ limit / 1024, (limit - mystat.stat[MCS_RSS]) / 1024,<br>
+ memsw_limit / 1024, (memsw_limit - swap_in_u) / 1024);<br>+<br>+ return 0;<br>+#undef K<br>+}<br> <br> static struct cftype mem_cgroup_files[] = {<br> {<br>@@ -2228,6 +2258,10 @@<br> .name = "stat",<br>
.read_map = mem_control_stat_show,<br> },<br>+ {<br>+ .name = "meminfo",<br>+ .read_seq_string = mem_cgroup_meminfo,<br>+ },<br> {<br> .name = "force_empty",<br>
.trigger = mem_cgroup_force_empty_write,<br><br>-- <br>Krzysztof Taraszka<br><br><div class="gmail_quote">2009/8/23 Daniel Lezcano <span dir="ltr"><<a href="mailto:daniel.lezcano@free.fr">daniel.lezcano@free.fr</a>></span><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="h5">Krzysztof Taraszka wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr" target="_blank">daniel.lezcano@free.fr</a>><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Krzysztof Taraszka wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/8/23 Krzysztof Taraszka <<a href="mailto:krzysztof.taraszka@gnuhosting.net" target="_blank">krzysztof.taraszka@gnuhosting.net</a>><br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/8/23 Krzysztof Taraszka <<a href="mailto:krzysztof.taraszka@gnuhosting.net" target="_blank">krzysztof.taraszka@gnuhosting.net</a>><br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr" target="_blank">daniel.lezcano@free.fr</a>><br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Krzysztof Taraszka wrote:<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
2009/8/23 Daniel Lezcano <<a href="mailto:daniel.lezcano@free.fr" target="_blank">daniel.lezcano@free.fr</a>><br>
<br>
Krzysztof Taraszka wrote:<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hello,<br>
<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I am running lxc on my debian unstable sandbox and I have a few<br>
question<br>
about memory managament inside linux containers based on lxc<br>
project.<br>
<br>
I have got linux kernel 2.6.30.5 with enabled :<br>
<br>
+Resource counter<br>
++ Memory Resource Controller for Control Groups<br>
+++ Memory Resource Controller Swap Extension(EXPERIMENTAL)<br>
<br>
lxc-checkconfig pass all checks.<br>
<br>
I read about cgroups memory managament<br>
(Documentation/cgroups/memory.txt)<br>
and I tried to pass those value to my debian sandbox.<br>
<br>
And...<br>
'free -m' and 'top/htop' still show all available memory inside<br>
container<br>
(also If I set 32M for lxc.cgroup.memory.limit_in_bytes and<br>
lxc.cgroup.memory.usage_in_bytes; and 64M for<br>
lxc.cgroup.memory.memsw.usage_in_bytes and<br>
lxc.cgroup.memory.memsw.limit_in_bytes free and top show all<br>
resources).<br>
<br>
What I did wrong? Does the container always show all available<br>
memory<br>
resources without cgroup limitations?<br>
<br>
At the first glance I would say the configuration is correct.<br>
<br>
<br>
</blockquote>
But AFAIR, the memory cgroup is not isolated, if you specify 32MB you<br>
will<br>
see all the memory available on the system either if you are not<br>
allowed to<br>
use more than 32MB. If you create a program which allocates 64MB<br>
within<br>
a<br>
container configured with 32MB, and you "touch" the pages (may be<br>
that<br>
can<br>
be done with one mmap call with the MAP_POPULATE option), you should<br>
see the<br>
application swapping and the "memory.failcnt" increasing.<br>
<br>
IMHO, showing all the memory available for the system instead of<br>
showing<br>
the allowed memory with the cgroup is weird but maybe there is a good<br>
reason<br>
to do that.<br>
<br>
<br>
<br>
<br>
<br>
</blockquote>
Thank you Daniel for your reply.<br>
I think that LXC should isolate memory available for containers like<br>
Vserver<br>
or FreeVPS do (memory + swap) if .cgroup.memory.* and<br>
lxc.cgroup.memory.memsw.* is set.<br>
Is there any possibility to make a patch for linux kernel / lxc-tools<br>
to<br>
show the limitations inside cointainers propertly? I think is a good<br>
idea<br>
and it should be apply as soon as possible.<br>
<br>
<br>
<br>
</blockquote>
Maybe a solution can be to add a new memory.meminfo file in the same<br>
format than /proc/meminfo, so it will be possible to mount --bind<br>
/cgroup/foo/memory.meminfo to /proc/meminfo for the container.<br>
<br>
<br>
<br>
</blockquote>
Yes, I thought the same. This should allow the user-space tools based on<br>
/proc/meminfo (such as comand line "free") show limited information :)<br>
<br>
<br>
<br>
</blockquote>
Hmmm... does the memory.stat is a good start point for make new one<br>
object<br>
memory.meminfo similar to /proc/meminfo? If so, I can play by my self<br>
with<br>
lxc-tools code.<br>
<br>
<br>
<br>
</blockquote>
<br>
Hmmm... Daniel, I have got a question (that do I thinking in the right<br>
way).<br>
here is an output from /proc/meminfo from openvz:<br>
<br>
<br>
MemTotal: 262144 kB<br>
MemFree: 232560 kB<br>
Buffers: 0 kB<br>
Cached: 0 kB<br>
SwapCached: 0 kB<br>
Active: 0 kB<br>
Inactive: 0 kB<br>
HighTotal: 0 kB<br>
HighFree: 0 kB<br>
LowTotal: 262144 kB<br>
LowFree: 232560 kB<br>
SwapTotal: 0 kB<br>
SwapFree: 0 kB<br>
Dirty: 0 kB<br>
Writeback: 0 kB<br>
AnonPages: 0 kB<br>
Mapped: 0 kB<br>
Slab: 0 kB<br>
SReclaimable: 0 kB<br>
SUnreclaim: 0 kB<br>
PageTables: 0 kB<br>
NFS_Unstable: 0 kB<br>
Bounce: 0 kB<br>
WritebackTmp: 0 kB<br>
CommitLimit: 0 kB<br>
Committed_AS: 0 kB<br>
VmallocTotal: 0 kB<br>
VmallocUsed: 0 kB<br>
VmallocChunk: 0 kB<br>
HugePages_Total: 0<br>
HugePages_Free: 0<br>
HugePages_Rsvd: 0<br>
HugePages_Surp: 0<br>
Hugepagesize: 2048 kB<br>
<br>
most of values are 0.<br>
<br>
I have an question about SwapTotal and SwapFree for LXC.<br>
As I thinking that:<br>
<br>
MemTotal might be: hierarchical_memory_limit<br>
MemFree might be: hierarchical_memory_limit - cache<br>
<br>
<br>
</blockquote>
I am not a memory expert, but isn't MemFree : hierarchical_memory_limit -<br>
rss ?<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
the<br>
<br>
SwapTotal might be: hierarchical_memsw_limit<br>
SwapFree might be: hierarchical_memsw_limit - rss<br>
<br>
rss - # of bytes of anonymous and swap cache memory<br>
I don't know at all that hierarchical_memsw_limit is an good value for<br>
swap<br>
total, because as I read it is a mem+swap at all.<br>
<br>
Does the lxc memory.meminfo might look like above? Where can I get the<br>
Hugepagesize?<br>
<br>
<br>
</blockquote>
Right, I agree most of the interesting information to create a<br>
memory.meminfo is already there in another file and another format. Probably<br>
some informations in memory.stat can be moved to memory.meminfo and this one<br>
can be step by step filled with cgroup memory statistic informations. IMO,<br>
if the memory controller displays memory statistics like a proc/meminfo file<br>
format, that will make consistency for these informations and make trivial<br>
the isolation/virtualization with a simple mount-bind.<br>
<br>
<br>
<br>
</blockquote>
Hmm..<br>
might be. Right now I am looking for and writing new function in<br>
mm/memcontrol.c file for writing some stats in memory.meminfo file for<br>
tests.<br>
Dirty and ugly part of code, but if it will work as we thought (mount-bind)<br>
and as you wrote above, that will be very simple.<br>
I am going to look how does the /proc/meminfo is doing by the openvz.<br>
mm/memcontrol.c was wrote by xemul from openvz and balbir from ibm.<br>
If I am thinking in the right way, guys from openvz made their own patch for<br>
meminfo based on the mm/memcontrol.c. If I am wrong - where do they taking<br>
meminfo data? :)<br>
</blockquote>
<br></div></div>
I did this ugly patch patch for MemTotal/MemFree - maybe wrong :)<br>
<br>
Index: linux-2.6/mm/memcontrol.c<br>
===================================================================<br>
--- linux-2.6.orig/mm/memcontrol.c 2009-06-23 12:00:52.000000000 +0200<br>
+++ linux-2.6/mm/memcontrol.c 2009-08-23 22:49:02.000000000 +0200<br>
@@ -2200,6 +2200,27 @@ static int mem_cgroup_swappiness_write(s<br>
}<br>
<br>
<br>
+static int mem_cgroup_meminfo(struct cgroup *cgrp, struct cftype *cft,<br>
+ struct seq_file *seq)<br>
+{<br>
+#define K(x) ((x) << 10)<br>
+<br>
+ struct mem_cgroup *mem_cont = mem_cgroup_from_cont(cgrp);<br>
+ struct mcs_total_stat mystat = { };<br>
+ unsigned long long limit, memsw_limit;<br>
+<br>
+ mem_cgroup_get_local_stat(mem_cont, &mystat);<br>
+ memcg_get_hierarchical_limit(mem_cont, &limit, &memsw_limit);<br>
+<br>
+ seq_printf(seq,<br>
+ "MemTotal: %8llu kB\n"<br>
+ "MemFree: %8llu kB\n",<br>
+ limit / 1024, (limit - mystat.stat[MCS_RSS]) / 1024);<br>
+<br>
+ return 0;<br>
+#undef K<br>
+}<br>
+<br>
static struct cftype mem_cgroup_files[] = {<br>
{<br>
.name = "usage_in_bytes",<br>
@@ -2242,6 +2263,10 @@ static struct cftype mem_cgroup_files[]<br>
.read_u64 = mem_cgroup_swappiness_read,<br>
.write_u64 = mem_cgroup_swappiness_write,<br>
},<br>
+ {<br>
+ .name = "meminfo",<br>
+ .read_seq_string = mem_cgroup_meminfo,<br>
+ },<br>
};<br>
<br>
#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP<br>
<br>
<br>
With the lxc tools I did:<br>
<br>
lxc-execute -n foo /bin/bash<br>
echo 268435456 > /cgroup/foo/memory.limit_in_bytes<br>
mount --bind /cgroup/foo/memory.meminfo /proc/meminfo<br>
for i in $(seq 1 100); do sleep 3600 & done<br>
<br>
And the result for "free" is:<br>
<br>
free:<br>
<br>
total used free shared buffers cached<br>
Mem: 262144 9692 252452 0 0 0<br>
-/+ buffers/cache: 9692 252452<br>
Swap: 0 0 0<br>
<br>
<br>
and for "top":<br>
<br>
top - 22:57:37 up 8 min, 1 user, load average: 0.00, 0.02, 0.00<br>
Tasks: 104 total, 1 running, 103 sleeping, 0 stopped, 0 zombie<br>
Cpu(s): 0.3%us, 1.0%sy, 0.0%ni, 98.4%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st<br>
Mem: 262144k total, 9864k used, 252280k free, 0k buffers<br>
Swap: 0k total, 0k used, 0k free, 0k cached<br>
<br>
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br>
337 root 20 0 14748 1132 872 R 1.0 0.4 0:00.24 top <br>
1 root 20 0 8136 484 408 S 0.0 0.2 0:00.00 lxc-init <br>
2 root 20 0 89980 1724 1348 S 0.0 0.7 0:00.70 bash <br>
25 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep <br>
232 root 20 0 86916 616 524 S 0.0 0.2 0:00.00 sleep <br>
233 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep <br>
234 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep <br>
235 root 20 0 86916 612 524 S 0.0 0.2 0:00.00 sleep <br>
.....<br>
<br>
<br>
:)<br>
<br>
</blockquote></div><br>