[lxc-devel] "pre-start" hooks and avahi

Sun Jan 12 16:39:15 UTC 2014

On Sun, Jan 12, 2014 at 10:21:15AM +0100, Chris Glass wrote:
> Further investigation (with trace):
> 
> It seems the failure I'm seeing is caused by not being able to talk to
> dbus's system bus in the pre-start hook [1].
> Looking at the code (src/lxc/start.c:355 ish), it seems pre-start
> hooks are called after lxc_cmd_start and seccomp have been
> initialised.
> I am expecting the pre-start hooks to be executed in a pristine host
> environment with some LXC variables. Is that wrong, an is it actually
> run in a (more) permissive guest jail?
> 
> Is that a bug, or do I have wrong expectations about pre-start hooks?
> 
> I would be glad to help either way - I'd love to follow that rabbit
> hole since it's been a great learning experience for me so far, or I
> can add to the docs to prevent other people to fall into the same trap
> :)
> 
> - Chris

I think I tracked down your problem:
Failed to open connection to system bus: An AppArmor policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_call", sender="(null)" (inactive) interface="org.freedesktop.DBus" member="Hello" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)

Basically Apparmor is preventing lxc-start from talking to dbus. We'll
have to change this for cgmanager anyway, so that should fix the problem
for you at the same time.

In the meantime, you can simply add "dbus," to
/etc/apparmor.d/abstractions/lxc/start-container and do
"/etc/init.d/apparmor reload", that should fix it.

> 
> 
> [1]: Minimal pre-start hook: http://pastebin.ubuntu.com/6737406/
> Resulting strace: http://pastebin.ubuntu.com/6737363/
> 
> On Sat, Jan 11, 2014 at 4:41 PM, Chris Glass <chris.glass at canonical.com> wrote:
> > Hi,
> >
> > Unfortunately I don't think that's the cause of this particular issue.
> > The error happens in *pre-start*, so the container is not yet started
> > as far as I can tell (the documentation stipulates that it's executed
> > in the host's environment).
> > My changes do not need avahi to be installed on the guests at all. I
> > want to query avahi from the host's environment for the info I need in
> > pre-start, and put that in a file in the container's rootfs, then boot
> > it normally.
> >
> > For the record I tried disabling rlimit_nproc in the host's avahi
> > daemon (as well as setting it to a ridiculous number like 200), but
> > that doesn't change anything.
> >
> > I'm not sure how to investigate this further... pointers welcome!
> >
> > - Chris
> >
> >
> > On Fri, Jan 10, 2014 at 7:47 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> >> Avahi sets its own rlimit_nproc to be exactly the # tasks it will spawn
> >> as its userid.  If the avahi user in the container is the same uid #
> >> as something in use on the host, then there may already be running
> >> tasks and so rlimit_nproc is enforced and avahi can't run.
> >>
> >> If you use a user namespace this shouldn't be an issue (so long as
> >> you don't share uid ranges between any containers).
> >>
> >> Otherwise, you have to change the avahi code to check whether it is in a
> >> container before setting rlimit_nproc.  There is a patch to work around
> >> this in the ubuntu trusty package, at least, if you wnat an example of
> >> how to do it.
> >>
> >> -serge
> >>
> >> Quoting Chris Glass (chris.glass at canonical.com):
> >>> Hi all,
> >>>
> >>> I'm currently hacking on a pre-start hook for the ubuntu and
> >>> ubuntu-cloud templates that automatically makes ubuntu containers
> >>> aware of squid-deb-proxy servers the host knows about.
> >>>
> >>> For this, I assume the squid-deb-proxy-client package is installed on
> >>> the host. If you're not familiar with this package: it's basically a
> >>> custom squid config for deb files along with an avahi config to expose
> >>> an _apt_proxy._tcp service pointing to the squid.
> >>> I currently run a squid-deb-proxy in a container.
> >>>
> >>> While running the avahi command on the host works fine (both as my
> >>> user and as root), it fails when run from a pre-start script, and my
> >>> knowledge of lxc is too limited to understand what could interfere
> >>> with it.
> >>>
> >>> The failure:
> >>> Running "avahi-browse -kprt _apt_proxy._tcp" in a lxc.hook.pre-start
> >>> hook fails with "Failed to create client object: Access denied". It
> >>> succeeds when ran at a normal shell on the host.
> >>>
> >>> A quick "whoami" in the same context yields "root", as expected, and
> >>> switching to an unconfined apparmor profile does not change anything,
> >>> so I suspect something more subtle is going on. How is the environment
> >>> on "pre-start" hooks different?
> >>>
> >>> Could somebody shed some light here?
> >>>
> >>> Background:
> >>> The reason for this is that I am a little frustrated to have to update
> >>> my LXC container's apt proxy settings every time my squid-deb-proxy
> >>> environment changes, and so this is an attempt at making this
> >>> automatic. If you are curious and want to see it for yourself, have a
> >>> look at [1], but please be aware that it's still work in progress.
> >>>
> >>> Thanks a lot for your help,
> >>>
> >>> - Chris
> >>>
> >>> links:
> >>> --------
> >>> [1]: https://github.com/chrisglass/lxc/blob/make-lxc-squid-deb-proxy-aware/hooks/squid-deb-proxy-client
> >>> _______________________________________________
> >>> lxc-devel mailing list
> >>> lxc-devel at lists.linuxcontainers.org
> >>> http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> _______________________________________________
> >> lxc-devel mailing list
> >> lxc-devel at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-devel
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140112/bd59efa0/attachment.pgp>