[lxc-users] Disappearing cgroups

[Serge E. Hallyn]

On Tue, Oct 08, 2019 at 02:34:23PM +0100, Ben Green wrote:
> Quoting "Serge E. Hallyn" <serge at hallyn.com>:
> 
> > > 
> > > I'm not sure it will be this, as the loss of cgroups happens at random
> > > and I've not been able to find an associated login/logout event. It did
> > 
> > Oh, yeah, then that probably isn't it.
> 
> Thanks for thinking about this, it so hard as this is stopping me using LXC
> in production and I really want to be able to do that. For more information
> I created a script to figure out where the container was running, which user
> started it and which cgroups were attached. I found the exact moment they
> disappeared:
> 
> 
> Tue  8 Oct 00:35:01 BST 2019
> 
> --
> Container: ex386-jessie
> Launch User: lxcadmin
> CGROUPS: devices freezer perf_event cpuset memory blkio cpu,cpuacct
> net_cls,net_prio systemd
> CGDIR: /sys/fs/cgroup/memory/user.slice/user-202.slice/session-3.scope/lxc.payload/ex386-jessie/
> RAM: 2048
> SWAP: 8796093022207
> 
> 
> --
> Container: ex64-jessie
> Launch User: root
> CGROUPS: devices freezer perf_event cpuset memory blkio cpu,cpuacct
> net_cls,net_prio systemd
> CGDIR: /sys/fs/cgroup/memory/user/lxcadmin/0/lxc.payload/ex64-jessie/
> RAM: 2048
> SWAP: 4096
> 
> 
> 
> 
> Tue  8 Oct 00:40:01 BST 2019
> 
> --
> Container: ex386-jessie
> Launch User: lxcadmin
> CGROUPS: freezer perf_event cpuset memory net_cls,net_prio systemd
> CGDIR: /sys/fs/cgroup/memory/user.slice/user-202.slice/session-3.scope/lxc.payload/ex386-jessie/
> RAM: 2048
> SWAP: 8796093022207
> 
> 
> --
> Container: ex64-jessie
> Launch User: CGROUPS: freezer perf_event cpuset net_cls,net_prio systemd
> CGDIR:
> RAM: SWAP:
> 
> 
> 
> 
> As you can after the change, my script can't figure out where the cgroups
> have gone in the second case. Thought the servers are still running and can
> still be logged into via ssh, I can no longer lxc-attach to them:
> 
> 
> lxcadmin at oyster:~$ lxc-attach ex386-jessie
> lxc-attach: ex386-jessie: cgroups/cgfsng.c: cgfsng_attach: 2115 No such file
> or directory - Failed to attach 2017 to /sys/fs/cgroup/cpu,cpuacct/user/lxcadmin/0/lxc.payload/ex386-jessie/cgroup.procs
> 
> lxcadmin at oyster:~$ lxc-attach ex64-jessie
> lxc-attach: ex64-jessie: cgroups/cgfsng.c: cgfsng_attach: 2115 No such file
> or directory - Failed to attach 2042 to /sys/fs/cgroup/cpu,cpuacct/user/lxcadmin/0/lxc.payload/ex64-jessie/cgroup.procs
> 
> 
> 
> The only thing unusual in the host logs I can find is this:
> 
> /var/log/syslog
> Oct  8 00:37:19 oyster PackageKit: daemon quit
> Oct  8 00:37:19 oyster systemd[1]: packagekit.service: Main process exited,
> code=killed, status=15/TERM
> Oct  8 00:37:19 oyster systemd[1]: packagekit.service: Succeeded.
> 
> Otherwise everything looks identical. The container logs show now unusual
> activity at all. I'm completely lost as to what to do.

So at this point, which part of the path

/sys/fs/cgroup/memory/user.slice/user-202.slice/session-3.scope/lxc.payload/ex386-jessie/

still exists?  Is session-3.scope gone?  Is user-202.slice still there?