[lxc-users] LXCFS and ProcPS Interaction Issue

[Robert Pendell]

On Thu, Jun 4, 2015 at 11:39 AM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> Quoting Robert Pendell (shinji at elite-systems.org):
>> On Mon, Jun 1, 2015 at 9:11 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
>> > Quoting Robert Pendell (shinji at elite-systems.org):
>> >> On Mon, Jun 1, 2015 at 8:42 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
>> >> > Quoting Robert Pendell (shinji at elite-systems.org):
>> >> >> First some basic information so you know my environment...
>> >> >> Host: Linode
>> >> >> OS: Ubuntu 14.04.2
>> >> >> Kernel: Host supplied 4.0.1 -- also tested against PV-Grub loaded 4.1-RC4
>> >> >> Container Privilage: Unprivileged.
>> >> >> shinji at icarus:/proc> apt-cache policy lxcfs
>> >> >> lxcfs:
>> >> >>   Installed: 0.7-0ubuntu2~ubuntu14.04.1~ppa1
>> >> >>   Candidate: 0.7-0ubuntu2~ubuntu14.04.1~ppa1
>> >> >>   Version table:
>> >> >>  *** 0.7-0ubuntu2~ubuntu14.04.1~ppa1 0
>> >> >>         500 http://ppa.launchpad.net/ubuntu-lxc/daily/ubuntu/
>> >> >> trusty/main amd64 Packages
>> >> >>         100 /var/lib/dpkg/status
>> >> >>
>> >> >> I literally just noticed this today.  When LXCFS is started and
>> >> >> running ProcPS throws a Floating Point Exception.  I do not know why
>> >> >> this is the case.  Oddly enough stopping LXCFS on the host and
>> >> >> restarting the container makes the bug go away.  Ideas?  Anything I
>> >> >> can test to isolate?
>> >> >
>> >> > Can you build 0.9.0 from the wily sources, just to make sure this isn't
>> >> > somethign fixed upstream?
>> >> >
>> >> > Also, if you can run lxcfs under gdb and get stack trace when it crashes
>> >> > that woudl be helpful.
>> >> >
>> >> >> LXCFS Stopped
>> >> >> shinji at trusty-x86:~> ps u
>> >> >> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> >> >> shinji    1506  0.0  0.1   5816  3536 pts/4    Ss+  12:42   0:00 -bash
>> >> >> shinji    1519  0.0  0.1   5404  2684 pts/4    S+   12:42   0:00 tmux -2 -f /usr
>> >> >> shinji    1573  0.3  0.1   5696  3412 pts/5    Ss   12:42   0:00 /bin/bash
>> >> >> shinji    1679  0.0  0.1   5228  2340 pts/5    R+   12:42   0:00 ps u
>> >> >>
>> >> >> LXCFS Started
>> >> >> shinji at trusty-x86:~> ps u
>> >> >> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> >> >> Signal 8 (FPE) caught by ps (procps-ng version 3.3.9).
>> >> >> shinji    1521  0.0ps:display.c:66: please report this bug
>> >> >> Floating point exception
>> >> >>
>> >>
>> >> I could but I guess I forgot to mention on here that the issue was
>> >> already identified when I ran a ticket on LXC.
>> >
>> > Ah, great, thanks.
>> >
>>
>> I went ahead and did that testing but I couldn't get lxcfs 0.9 to work
>> at all.  It built but it didn't seem to respond nor did LXC seem to
>> attempt to use it.  I removed lxcfs 0.7 before testing and had it
>> install in the default location (/usr/local).  At no time did it seem
>
> You probably should have used /usr.  Check where lxcfs 0.9 is putting it's
> fs - presumably it's under /usr/local/var/lib/lxcfs instead of
> /var/lib/lxcfs.  The lxcfs lxc hook may be trying to bind-mount from
> /var/lib.
>
>> like the container was using lxcfs as all /proc contents were a mirror
>> of the host.
>>
>> However I did check /var/lib/lxcfs/proc and found that, once again,
>
> (oh, yes, there it is)
>
>> meminfo was blank so it looks like that the issue does indeed persist
>
> Hm.
>
> Can you show your /proc/meminfo?

Do you want the meminfo from the host or the container?

>
>> regardless.  The stacktrace was non-conclusive given that we already
>> know the scenario that needs handled/patched.
>
> Do we?  Sorry i don't seem to have that context handy.  Which scenario
> is it?

As per my ticket on githib LXCFS fails to generate an appropriate
/proc/meminfo for container use in the event that the cgroup memory
controller is unavailable.  This may be either because it wasn't
compiled in or it is disabled at boot.

The memory cgroup isn't compiled into the kernel being used at all
The memory cgroup functions are compiled in but are disabled at boot
time (cgroup_disable=memory)
The memory cgroup functions are compiled in but are disabled by
default at boot time

I had done some additional testing before submitting the ticket and
when I had the memory cgroup available lxcfs had successfully
generated an appropriate /proc/meminfo that my containers used.

For me I fall under the first scenario.