[Lxc-users] can't restart container without rebooting entire host, because can't delete cgroups files, tasks is 0

[Daniel Lezcano]

On 11/15/2010 03:26 PM, Miroslav Lednicky, AVONET, s.r.o. wrote:
> Hello,
>
>     please see:
>
> ls -l
> total 0
> drwxr-xr-x 4 root root 0 2010-11-15 15:00 1285
> drwxr-xr-x 4 root root 0 2010-11-15 15:00 1298
> drwxr-xr-x 4 root root 0 2010-11-15 15:01 1322
> drwxr-xr-x 4 root root 0 2010-11-15 15:01 1325
> drwxr-xr-x 4 root root 0 2010-11-15 15:02 1335
> drwxr-xr-x 4 root root 0 2010-11-15 15:09 1386
> drwxr-xr-x 4 root root 0 2010-11-15 15:11 1401
> drwxr-xr-x 4 root root 0 2010-11-15 15:12 1408
> drwxr-xr-x 4 root root 0 2010-11-15 15:12 1411
> drwxr-xr-x 3 root root 0 2010-11-15 15:17 1459
> -r--r--r-- 1 root root 0 2010-11-15 15:02 cgroup.procs
> -r--r--r-- 1 root root 0 2010-11-15 15:02 cpuacct.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuacct.usage
> -r--r--r-- 1 root root 0 2010-11-15 15:02 cpuacct.usage_percpu
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpu.rt_period_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpu.rt_runtime_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.cpu_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.cpus
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.mem_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.mem_hardwall
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.memory_migrate
> -r--r--r-- 1 root root 0 2010-11-15 15:02 cpuset.memory_pressure
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.memory_spread_page
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.memory_spread_slab
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.mems
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.sched_load_balance
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpuset.sched_relax_domain_level
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 cpu.shares
> --w------- 1 root root 0 2010-11-15 15:02 devices.allow
> --w------- 1 root root 0 2010-11-15 15:02 devices.deny
> -r--r--r-- 1 root root 0 2010-11-15 15:02 devices.list
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 freezer.state
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.failcnt
> --w------- 1 root root 0 2010-11-15 15:02 memory.force_empty
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.max_usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.memsw.failcnt
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.memsw.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.memsw.max_usage_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:02 memory.memsw.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.soft_limit_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:02 memory.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.swappiness
> -r--r--r-- 1 root root 0 2010-11-15 15:02 memory.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 memory.use_hierarchy
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 net_cls.classid
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 notify_on_release
> -rw-r--r-- 1 root root 0 2010-11-15 15:02 tasks
> root at lnx-zl-teaspl:/cgroup/teas_www# ls -lR 1285
> 1285:
> total 0
> drwxr-xr-x 2 root root 0 2010-11-15 15:00 2
> drwxr-xr-x 2 root root 0 2010-11-15 15:00 3
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cgroup.procs
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage_percpu
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_period_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_runtime_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpu_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpus
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_hardwall
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_migrate
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_pressure
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_page
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_slab
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mems
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_load_balance
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_relax_domain_level
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.shares
> --w------- 1 root root 0 2010-11-15 15:00 devices.allow
> --w------- 1 root root 0 2010-11-15 15:00 devices.deny
> -r--r--r-- 1 root root 0 2010-11-15 15:00 devices.list
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 freezer.state
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.failcnt
> --w------- 1 root root 0 2010-11-15 15:00 memory.force_empty
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.max_usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.failcnt
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.max_usage_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.soft_limit_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.swappiness
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.use_hierarchy
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 net_cls.classid
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 notify_on_release
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 tasks
>
> 1285/2:
> total 0
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cgroup.procs
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage_percpu
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_period_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_runtime_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpu_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpus
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_hardwall
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_migrate
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_pressure
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_page
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_slab
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mems
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_load_balance
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_relax_domain_level
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.shares
> --w------- 1 root root 0 2010-11-15 15:00 devices.allow
> --w------- 1 root root 0 2010-11-15 15:00 devices.deny
> -r--r--r-- 1 root root 0 2010-11-15 15:00 devices.list
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 freezer.state
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.failcnt
> --w------- 1 root root 0 2010-11-15 15:00 memory.force_empty
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.max_usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.failcnt
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.max_usage_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.soft_limit_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.swappiness
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.use_hierarchy
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 net_cls.classid
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 notify_on_release
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 tasks
>
> 1285/3:
> total 0
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cgroup.procs
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuacct.usage_percpu
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_period_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.rt_runtime_us
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpu_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.cpus
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_exclusive
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mem_hardwall
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_migrate
> -r--r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_pressure
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_page
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.memory_spread_slab
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.mems
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_load_balance
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpuset.sched_relax_domain_level
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 cpu.shares
> --w------- 1 root root 0 2010-11-15 15:00 devices.allow
> --w------- 1 root root 0 2010-11-15 15:00 devices.deny
> -r--r--r-- 1 root root 0 2010-11-15 15:00 devices.list
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 freezer.state
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.failcnt
> --w------- 1 root root 0 2010-11-15 15:00 memory.force_empty
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.max_usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.failcnt
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.limit_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.max_usage_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.memsw.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.soft_limit_in_bytes
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.stat
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.swappiness
> -r--r--r-- 1 root root 0 2010-11-15 15:00 memory.usage_in_bytes
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 memory.use_hierarchy
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 net_cls.classid
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 notify_on_release
> -rw-r--r-- 1 root root 0 2010-11-15 15:00 tasks
>
> It is contentof my cgroup directory with running LXC. There are
> directories:
>
> drwxr-xr-x 4 root root 0 2010-11-15 15:00 1285
> drwxr-xr-x 4 root root 0 2010-11-15 15:00 1298
> drwxr-xr-x 4 root root 0 2010-11-15 15:01 1322
> drwxr-xr-x 4 root root 0 2010-11-15 15:01 1325
> drwxr-xr-x 4 root root 0 2010-11-15 15:02 1335
> drwxr-xr-x 4 root root 0 2010-11-15 15:09 1386
> drwxr-xr-x 4 root root 0 2010-11-15 15:11 1401
> drwxr-xr-x 4 root root 0 2010-11-15 15:12 1408
> drwxr-xr-x 4 root root 0 2010-11-15 15:12 1411
> drwxr-xr-x 3 root root 0 2010-11-15 15:17 1459
>
> These PIDs are no in global proc filesystem
> and still increasing.
>
> Load of my machine is still higher and higher.
>
> I will must restart computer ant the end. :-(
>
> What can i do with it?

Ok, let's try to understand.

Let's do it step-by-step :

1 - the topmost directory shows "freezer.state" and it shouldn't because 
you can not freeze the system (except if there is a recent change in the 
kernel)

2 - try to delete the 1285/2 directory, if you can't check the content 
of 1285/2/tasks and look for the process in the system: is it a process 
running inside a container ?

Oh, a dumb question : are you using the libvirt with lxc ?