[lxc-users] Better error logging when starting containers?

[John Siu]

I can give you the issue I filed with lxc/lxc as an example : https://github.com/lxc/lxc/issues/819

If you try to create a container using the following command:

	lxc-create -t download -n lxc10009 -f lxc10009.conf -- -d ubuntu -r xenial -a amd64

And the config file contain a “lxc.roofs” line, it will failed with following error:

	lxc-create: lxc_create.c: main: 303 Error creating container lxc10009

I can’t determine the cause of the error even I went into the code (lxc_create.c, line 303).

I end up commenting the config file section by section, line by line to find the offending line.

Another example is if you add network interfaces to an unprivileged container and forget to add that nic in /etc/lxc/lxc-usernet, the fail to start error messages contain very little hint about it. Even with “-l 9”, you will only get the following:

      lxc-start 20160222151046.765 ERROR    lxc_start - start.c:lxc_spawn:1108 - failed to create the configured network
      lxc-start 20160222151046.765 ERROR    lxc_start - start.c:__lxc_start:1274 - failed to spawn 'lxc10001'
      lxc-start 20160222151052.301 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160222151052.301 ERROR    lxc_start_ui - lxc_start.c:main:346 - To get more details, run the container in foreground mode.
      lxc-start 20160222151052.301 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.

So is it a network interface error on the host? Or lxc10001 config file error in the network line? Is it ip conflict? Bla bla bla … You get the idea. I personally wasted a lot of time on this particular one :(

Currently I take that as “growing pain” as all packages, technologies(lxc, cgroup, systemd) surrounding Linux container are all evolving rapidly. However I hope this issue can be improved faster. Currently it is difficult for end user (non-developer), or even developers but not actively involve in those packages, to determine the cause (config error? actual bug?) with those error messages.

John

> On Feb 22, 2016, at 14:11, Akshay Karle <akshay.a.karle at gmail.com> wrote:
> 
> I agree with your comments and the fact that the team is busy working lxc v2, but I wanted to get a sense of whether it was a problem everyone is facing and that the lxc team are aware of. I didn't look into the codebase for logging yet and I'm not a C programmer (anymore), but I would like to give a shot at improving the logs and hence asked for your suggestions. I will begin by looking at how we can improve error logging for lxc-start at least and then look at the general lxc-* commands. Thanks for the comments Bostjan, I'll keep that in mind when I look through the code.
> 
> Thanks for the tip on bumping the log level for ephemeral containers John!
> 
> On Mon, Feb 22, 2016 at 4:41 AM Bostjan Skufca <bostjan at a2o.si <mailto:bostjan at a2o.si>> wrote:
> Dear Akshay,
> 
> I do agree with you and find this behaviour a bit annoying, yet I believe "patches welcome" response will follow shortly :)
> 
> On a more serious note:
> As I skimmed over LXC code a while ago, it seems LXC bails out on first error that occurs. This means that implementing your suggestion would simply mean keeping last error stored somewhere and displaying it before exiting lxc-start itself.
> 
> This would be solution for lxc-start, which you (and I) are probably the most interested in. Some more generic solution for all lxc-* tools should probably be more adequate, but that would need attention from one of the maintainers/core devs.
> 
> b.
> 
> On 22 February 2016 at 00:50, Akshay Karle <akshay.a.karle at gmail.com <mailto:akshay.a.karle at gmail.com>> wrote:
> Hello lxc users,
> 
> After having used lxc for a while now, I've realized that when the container fails to start, it fails with a very generic message as follows:
> 
> $ lxc-start -n test
> lxc-start: lxc_start.c: main: 344 The container failed to start.
> lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
> lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.
> 
> And if you are using ephemeral containers, the error is even more generic and with no way to increase the log level:
> 
> $ lxc-start-ephemeral -n e1 -o test -d
> setting rootfs to .%s. /home/vagrant/.local/share/lxc/e1/rootfs
> The container 'e1' failed to start.
> 
> I was wondering if someone felt the need of having a little more meaningful error messages giving a summary of the error in the console output. The container logfile does have way more descriptive error messages, but since you don't directly have the errors in the console output, you are forced to open the logfiles everytime something goes wrong. Instead if you had an output that just included a few important error lines from the logfile such as the following example:
> 
> $ lxc-start -n test
> lxc-start: lxc_start.c: main: 344 The container failed to start.
> lxc_start - start.c:lxc_spawn:1031 - failed creating cgroups
> lxc-start: lxc_start.c: main: 346 To get more details, run the container in foreground mode.
> lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.
> 
> Do you think this would help? Although I have no idea if this is simple to implement, I just wanted to get your ideas, suggestions and concerns (if any) before attempting to figure out a solution.
> 
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org <mailto:lxc-users at lists.linuxcontainers.org>
> http://lists.linuxcontainers.org/listinfo/lxc-users <http://lists.linuxcontainers.org/listinfo/lxc-users>
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org <mailto:lxc-users at lists.linuxcontainers.org>
> http://lists.linuxcontainers.org/listinfo/lxc-users <http://lists.linuxcontainers.org/listinfo/lxc-users>_______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20160222/23e2a46b/attachment.html>