[lxc-users] Cgroup saga continues - lxc-autostart woes.

[Ben Green]

On 22/11/2020 15:09, Serge E. Hallyn wrote:
> Can you cat /proc/$pid/cgroup for one of the tasks in an autostarted container,
> and show the /var/lib/lxc/container-name/config ?

I must apologise for such a later response, this is the first time I've 
been back to this problem since you responded.

I'm running as an unprivileged user, lxcadmin, so the configs are at 
/home/lxcadmin/.local/share/lxc/slug/config :

# "Secure" mounting
lxc.mount.auto = proc:mixed sys:ro cgroup:mixed
lxc.rootfs.path = dir:/home/lxcadmin/.local/share/lxc/slug/rootfs

# Common configuration
lxc.include = /usr/share/lxc/config/debian.common.conf

# Container specific configuration
lxc.tty.max = 4
lxc.uts.name = slug
lxc.arch = amd64
lxc.pty.max = 1024

lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.memory.memsw.limit_in_bytes = 6G
lxc.mount.entry = none tmp tmpfs size=1024m,mode=1777 0 0
lxc.cap.drop = mac_admin mac_override net_admin sys_admin sys_module 
sys_rawio sys_time syslog sys_resource setpcap

# Subuids and subgids mapping
lxc.idmap = u 0 1258512 65536
lxc.idmap = g 0 1258512 65536
lxc.start.auto = 1

I've omitted the inferface settings. I've actually found that once 
again, containers are jumping out of their cgroups, 
/sys/fs/cgroup/memory/forcontainers/lxc/slug/tasks is again empty. I 
don't know how or why this has come about. This is now an absolute 
nightmare. We were hoping for this LXC setup to become a production 
setup and we've got a setup in which the containers get removed on a 
regular yet seemingly motiveless basis. I've put these containers into 
the forcontainers group as suggested. The below does seem very revealing 
though.

Here's and example PID from inside the server which has just started:

11:net_cls,net_prio:/
10:cpu,cpuacct:/../../../system.slice/lxc-start-lxcadmin-servers.service
9:cpuset:/
8:rdma:/
7:perf_event:/
6:memory:/../../../system.slice/lxc-start-lxcadmin-servers.service
5:pids:/../../../system.slice/lxc-start-lxcadmin-servers.service
4:freezer:/
3:blkio:/../../../system.slice/lxc-start-lxcadmin-servers.service
2:devices:/../../../system.slice/lxc-start-lxcadmin-servers.service
1:name=systemd:/
0::/

Here's one which has lost it's pids in a unknown process which happens 
seemingly randomly:

11:rdma:/
10:freezer:/
9:perf_event:/
8:pids:/../../../user.slice/user-202.slice/session-59466.scope
7:blkio:/../../../user.slice
6:cpuset:/
5:cpu,cpuacct:/../../../user.slice
4:memory:/../../../user.slice/user-202.slice/session-59466.scope
3:net_cls,net_prio:/
2: :/../../../user.slice
1:name=systemd:/
0::/

This is one behaving as we would like, freshly started using lxc-start 
from and ssh session of the user lxcadmin.

11:net_cls,net_prio:/
10:cpu,cpuacct:/
9:cpuset:/
8:rdma:/
7:perf_event:/
6:memory:/
5:pids:/
4:freezer:/
3:blkio:/
2:devices:/
1:name=systemd:/
0::/

It's that pesky systemd isn't it? I've made a script to get things back 
to how we want which looks like this for the guest ${1}:

for j in pids blkio cpu,cpuacct memory devices; do
    for i in $(cat /sys/fs/cgroup/cpuset/forcontainers/lxc/${1}/tasks); do
       echo "${i}" >> /sys/fs/cgroup/${j}/forcontainers/lxc/${1}/tasks
    done
done

Cheers,

Ben