[lxc-users] CGManager and LXCFS causing lxc-start to fail for unprivileged containers

[Akshay Karle]

So after I removed the '-c freezer,..' part from the pam_cgm.so line and
trying to start the container, it switched the cgroup driver to cgmanager
from cgfs.

I noticed when I had just added all the controllers in the pam_cgm.so line
the output of /proc/self/cgroup was something like this:
11:name=systemd:/user/1000.user/48.session
10:perf_event:/user/test/0
9:memory:/user/test/0
8:hugetlb:/user/test/0
7:freezer:/user/test/0
6:devices:/user/test/0
5:cpuset:/user/test/0
4:cpuacct:/user/test/0
3:cpu:/user/test/0
2:blkio:/user/test/0

And after removing the -c from pam_cgm.so, the output of /proc/self/cgroup
was:
11:name=systemd:/user/test/0
10:perf_event:/user/test/0
9:memory:/user/test/0
8:hugetlb:/user/test/0
7:freezer:/user/test/0
6:devices:/user/test/0
5:cpuset:/user/test/0
4:cpuacct:/user/test/0
3:cpu:/user/test/0
2:blkio:/user/test/0

So it seemed to have switched to cgfs when the name controller was pointing
to the cgroup of the wrong user.

However, after removing -c from pam_cgm.so line and switching back to the
cgmanager driver, the container still fails to start but now the error is
different. The error looks something like this:

$ cat start-no-args.log | egrep -e 'ERROR|WARN'
      lxc-start 1454105667.463 WARN     lxc_confile -
confile.c:config_pivotdir:1801 - lxc.pivotdir is ignored.  It will soon
become an error.
      lxc-start 1454105667.464 WARN     lxc_cgmanager -
cgmanager.c:cgm_get:989 - do_cgm_get exited with error
      lxc-start 1454105667.471 ERROR    lxc_utils -
utils.c:setproctitle:1455 - Invalid argument - setting cmdline failed
      lxc-start 1454105667.556 WARN     lxc_cgmanager -
cgmanager.c:lxc_cgmanager_enter:693 - call to cgmanager_move_pid_sync
failed: invalid request
      lxc-start 1454105667.556 ERROR    lxc_start -
start.c:__lxc_start:1213 - failed to spawn 'test'
      lxc-start 1454105668.094 WARN     lxc_commands -
commands.c:lxc_cmd_rsp_recv:172 - command get_init_pid failed to receive
response
      lxc-start 1454105668.095 WARN     lxc_cgmanager -
cgmanager.c:cgm_get:989 - do_cgm_get exited with error
      lxc-start 1454105673.099 ERROR    lxc_start_ui - lxc_start.c:main:344
- The container failed to start.
      lxc-start 1454105673.100 ERROR    lxc_start_ui - lxc_start.c:main:346
- To get more details, run the container in foreground mode.
      lxc-start 1454105673.100 ERROR    lxc_start_ui - lxc_start.c:main:348
- Additional information can be obtained by setting the --logfile and
--logpriority options.

You can find the complete lxc-start logs attached.

On Fri, Jan 29, 2016 at 4:33 PM Akshay Karle <akshay.a.karle at gmail.com>
wrote:

> Hey Serge,
>
> Note, just dropping the '-c freezer' argument also will tell pam_cgm.so
>> to use all controllers.
>>
>
> That's good to know. Just tried it out, it works. Thanks!
>
>>
>> The debug info above says lxc is using cgfs and not cgmanager.  Exactly
>> which lxc package version are you using?
>>
>
> I'm using lxc 1.1.5. Exact version 1.1.5-0ubuntu5~ubuntu14.04.1~ppa1
> available from ppa:ubuntu-lxc/stable. Reading through the container start
> logs I had sent in the previous email I found that it is indeed using
> cgroupfs driver. Is this the cause of the problem? Should I be running cgfs
> on trusty to begin with?
>
> Looking at the code of cgroup.c
> <https://github.com/lxc/lxc/blob/lxc-1.1.5/src/lxc/cgroup.c#L48>, I saw
> that in the init it checks for HAVE_CGMANAGER macro but I can't seem to
> figure out where it is being initialized, I'm guessing it should be
> somewhere in the configure.ac but not sure. My C skills are really rusty
> now.
>
>
> _______________________________________________
>> lxc-users mailing list
>> lxc-users at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160129/12e7f1b8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: start-no-args.log
Type: application/octet-stream
Size: 6751 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160129/12e7f1b8/attachment-0001.obj>