<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I don’t have a way to track the memory usage of a container yet, but this issue seems very consistent among the containers. In fact, all containers have a higher than expected swap usage.<div class=""><br class=""></div><div class=""><br class=""></div><div class="">As a quick test, I modified the container profile to see if removing the swap and memory limits would help (removed the “lxc.cgroup.memory.memsw.limit_in_bytes” setting and changed "limits.memory.swap=false” ). The odd thing now is the available memory to the container seems to be maxed out at the limits.memory setting and does not include swap:</div><div class=""><br class=""></div><div class="">Top output from container</div><div class="">-----------------------------</div><div class=""><div class="">top - 09:48:50 up 9 min, 0 users, load average: 0.62, 0.67, 0.72</div><div class="">Tasks: 20 total, 1 running, 19 sleeping, 0 stopped, 0 zombie</div><div class="">%Cpu(s): 2.7 us, 3.8 sy, 0.0 ni, 93.2 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st</div><div class="">KiB Mem : 332800 total, 52252 free, 99324 used, 181224 buff/cache</div><div class="">KiB Swap: 19737596 total, 19737596 free, 0 used. 52252 avail Mem</div></div><div class="">-----------------------------</div><div class=""><br class=""></div><div class="">Notice the "52252 avail Mem”. Seems the container is maxed out at 325MB despite having access to 19G swap.</div><div class=""><br class=""></div><div class="">Confusing…</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><div><div class="">On Jun 6, 2017, at 5:40 AM, Fajar A. Nugraha <<a href="mailto:list@fajar.net" class="">list@fajar.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote">On Tue, Jun 6, 2017 at 4:29 PM, Ron Kelley<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:rkelleyrtp@gmail.com" target="_blank" class="">rkelleyrtp@gmail.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">(Similar to a redit post:<span class="Apple-converted-space"> </span><a href="https://www.reddit.com/r/LXD/comments/53l7on/how_does_lxd_manage_swap_space" rel="noreferrer" target="_blank" class="">https://www.reddit.com/r/LXD/<wbr class="">comments/53l7on/how_does_lxd_<wbr class="">manage_swap_space</a>).<br class=""><br class="">Ubuntu 16.04, LXC 2.13 running about 50 containers. System has 8G RAM and 20G swap. From what I can tell, the containers are using lots of swap despite having free memory.<br class=""><br class=""><br class="">Top output from the host:<br class="">------------------------------<wbr class="">-<br class="">top - 05:23:24 up 15 days, 4:25, 2 users, load average: 0.29, 0.45, 0.62<br class="">Tasks: 971 total, 1 running, 970 sleeping, 0 stopped, 0 zombie<br class="">%Cpu(s): 8.9 us, 8.1 sy, 0.0 ni, 81.0 id, 0.7 wa, 0.0 hi, 1.2 si, 0.0 st<br class="">KiB Mem : 8175076 total, 284892 free, 2199792 used, 5690392 buff/cache<br class="">KiB Swap: 19737596 total, 15739612 free, 3997984 used. 3599856 avail Mem<br class="">------------------------------<wbr class="">-<br class=""><br class=""><br class="">Top output from a container:<br class="">------------------------------<wbr class="">-<br class="">top - 09:19:47 up 10 min, 0 users, load average: 0.52, 0.61, 0.70<br class="">Tasks: 17 total, 1 running, 16 sleeping, 0 stopped, 0 zombie<br class="">%Cpu(s): 0.3 us, 2.2 sy, 0.0 ni, 96.5 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st<br class="">KiB Mem : 332800 total, 148212 free, 79524 used, 105064 buff/cache<br class="">KiB Swap: 998400 total, 867472 free, 130928 used. 148212 avail Mem<br class="">------------------------------<wbr class="">-<br class=""><br class=""></blockquote><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Do you have history of memory usage inside the container? It's actually normal for linux to keep some elements in cache (e.g. inode entries), while forcing out program memory to swap. I'm guessing that's what happened during "busy" times, but now you see the non-busy times. Linux won't automatically put entries in swap back to memory if it's not used.</div><div class=""><br class=""></div><div class="">In the past I had to set vm.vfs_cache_pressure = 1000 to make linux release inode cache from memory. This was especially important on servers with lots of files. Nowadays I simply don't use swap anymore.</div><div class=""><br class=""></div><div class="">-- </div><div class="">Fajar</div></div></div></div><span style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">lxc-users mailing list</span><br style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:lxc-users@lists.linuxcontainers.org" style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">lxc-users@lists.linuxcontainers.org</a><br style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="http://lists.linuxcontainers.org/listinfo/lxc-users" style="font-family: Consolas; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">http://lists.linuxcontainers.org/listinfo/lxc-users</a></div></div><br class=""></div></body></html>