[lxc-users] nested containers
Mohan G
mohan_gg at yahoo.com
Fri Mar 6 10:10:04 UTC 2015
Thanks much.. This helps
RegardsMohan
From: Fajar A. Nugraha <list at fajar.net>
To: LXC users mailing-list <lxc-users at lists.linuxcontainers.org>
Sent: Friday, March 6, 2015 12:30 PM
Subject: Re: [lxc-users] nested containers
This is on ubuntu 14.10, lxc and lxcfs from ppa:ubuntu-lxc/daily
Test memory limit on the parent container, by writing data to /run/shm
(a tmpfs mount):
@host # lxc-cgroup -n v memory.use_hierarchy
1
@host # lxc-cgroup -n v memory.limit_in_bytes 1G
@host # lxc-cgroup -n v memory.limit_in_bytes
1073741824
@host # lxc-attach -n v -- mount | grep shm
none on /run/shm type tmpfs (rw,nosuid,nodev,relatime)
@host # lxc-cgroup -n v memory.usage_in_bytes
12881920
@host # lxc-attach -n v -- dd if=/dev/zero of=/run/shm/testfile
dd: writing to ‘/run/shm/testfile’: Cannot allocate memory
2076473+0 records in
2076472+0 records out
1063153664 bytes (1.1 GB) copied, 2.18223 s, 487 MB/s
@host # lxc-cgroup -n v memory.usage_in_bytes
1073561600
See how it's limited to around 1GB? Now delete the file before running
more test to free up memory.
@host # lxc-attach -n v -- rm /run/shm/testfile
@host # lxc-cgroup -n v memory.usage_in_bytes
10219520
Now test the limit on the nested container. We test with memory limit
2G. It should max out at 1G, since the parent only has that much
limit.
@host # lxc-attach -n v
root at v:~# lxc-cgroup -n nv memory.limit_in_bytes 2G
root at v:~# lxc-cgroup -n nv memory.limit_in_bytes
2147483648
root at v:~# lxc-cgroup -n nv memory.usage_in_bytes
7045120
root at v:~# lxc-attach -n nv -- dd if=/dev/zero of=/run/shm/testfile
dd: writing to ‘/run/shm/testfile’: Cannot allocate memory
2080265+0 records in
2080264+0 records out
1065095168 bytes (1.1 GB) copied, 2.96393 s, 359 MB/s
root at v:~# lxc-cgroup -n nv memory.usage_in_bytes
lxc_container: lxc_cgroup.c: main: 113 failed to retrieve value of
'memory.usage_in_bytes' for '/var/lib/lxc:nv'
root at v:~# lxc-ls -f
Killed
See how the testfile on the nested container is also limited at around
1GB? Note that at this time the container "v" (the parent) can't do
anything (it can't even run "lxc-ls"), because it wants to allocate
more memory, and all available memory has been used by the child
container "nv".
Since we can't run anything on the parent container due to memory
limit, kill the nested container
root at v:~# lxc-stop -k -n nv
lxc-stop: commands.c: lxc_cmd_stop: 615 failed to stop 'nv': Operation
not permitted
root at v:~# lxc-stop -k -n nv
nv is not running
root at v:~# lxc-ls -f
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
--------------------------------------------
nv STOPPED - - - NO
So in short, the nested limit works. To make it operable, make sure
that the parent container ALWAYS have some free memory to work with.
--
Fajar
On Fri, Mar 6, 2015 at 1:19 PM, Mohan G <mohan_gg at yahoo.com> wrote:
> Hi Folks,
> Let me explain my problem and then you can suggest me some way of over
> coming this.
> I wan to be able to run different protocol clients to use my file system
> mounted on the host. But i want to be able to limit their memory resource to
> 10Gb in total. Also, be able to set memory.sw limit so that each can have a
> softy limit of 5G. But if only one container is runing, it can use the
> entire 10G. I am able to use memory.use_hierarchy in cgroups to acheive the
> same. But how do i acheive this with container.
> Will nesting of containers help. my plan is to create a parent container and
> set limits to this and expect the nested conatiners to inherit these
> limit, and then i will set soft limits on these children container. But when
> i start nested containers, i don;t see any entrry under /sys/fs/groups for
> the children container.
>
> Regards
> Mohan
>
>
> ________________________________
> From: Fajar A. Nugraha <list at fajar.net>
> To: LXC users mailing-list <lxc-users at lists.linuxcontainers.org>
> Sent: Wednesday, March 4, 2015 2:44 PM
> Subject: Re: [lxc-users] nested containers
>
> On Wed, Mar 4, 2015 at 12:15 PM, Mohan G <mohan_gg at yahoo.com> wrote:
>
>
>
>> Hi,
>> Is there anyway we can have nested containers/cgroups. One parent
>> container
>> forming the basis for children containers. i.e subset of parent container.
>
>
> Yes.
>
> On parent container config (in ubuntu), add this:
> lxc.aa_profile=lxc-container-default-with-nesting
>
> And then on that container, you can create containers
>
> utopic ~ # lxc-ls -f --running
> NAME STATE IPV4 IPV6 GROUPS AUTOSTART
> -----------------------------------------------------------------
> v RUNNING 10.0.3.1, 192.168.124.173 - - NO
>
> utopic ~ # lxc-attach -n v
>
> root at v:~#
>
> root at v:~# cat /proc/1/cgroup
> 12:name=systemd:/lxc/v
> 11:perf_event:/lxc/v
> 10:net_prio:/lxc/v
> 9:net_cls:/lxc/v
> 8:memory:/lxc/v
> 7:hugetlb:/lxc/v
> 6:freezer:/lxc/v
> 5:devices:/lxc/v
> 4:cpuset:/lxc/v
> 3:cpuacct:/lxc/v
> 2:cpu:/lxc/v
> 1:blkio:/lxc/v
>
> root at v:~# lxc-create -t download -n nv -- -d ubuntu -r vivid -a amd64
> Using image from local cache
> Unpacking the rootfs
>
> ---
> You just created an Ubuntu container (release=vivid, arch=amd64,
> variant=default)
>
> To enable sshd, run: apt-get install openssh-server
>
> For security reason, container images ship without user accounts
> and without a root password.
>
> Use lxc-attach or chroot directly into the rootfs to set a root password
> or create user accounts.
>
> root at v:~# lxc-start -n nv
>
> root at v:~# lxc-ls -f --running
> NAME STATE IPV4 IPV6 GROUPS AUTOSTART
> --------------------------------------------------
> nv RUNNING 10.0.3.249 - - NO
>
>
>
> Now run a process inside the nested container
>
> root at v:~# lxc-attach -n nv -- cat /proc/1/cgroup
> 12:name=systemd:/lxc/v/lxc/nv
> 11:perf_event:/lxc/v/lxc/nv
> 10:net_prio:/lxc/v/lxc/nv
> 9:net_cls:/lxc/v/lxc/nv
> 8:memory:/lxc/v/lxc/nv
> 7:hugetlb:/lxc/v/lxc/nv
> 6:freezer:/lxc/v/lxc/nv
> 5:devices:/lxc/v/lxc/nv
> 4:cpuset:/lxc/v/lxc/nv
> 3:cpuacct:/lxc/v/lxc/nv
> 2:cpu:/lxc/v/lxc/nv
> 1:blkio:/lxc/v/lxc/nv
>
> Note how the cgroup is nested
>
> --
> Fajar
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20150306/514a8c29/attachment.html>
More information about the lxc-users
mailing list