[lxc-devel] "pre-start" hooks and avahi

Sun Jan 12 16:41:55 UTC 2014

On Sun, Jan 12, 2014 at 11:39:15AM -0500, Stéphane Graber wrote:
> On Sun, Jan 12, 2014 at 10:21:15AM +0100, Chris Glass wrote:
> > Further investigation (with trace):
> > 
> > It seems the failure I'm seeing is caused by not being able to talk to
> > dbus's system bus in the pre-start hook [1].
> > Looking at the code (src/lxc/start.c:355 ish), it seems pre-start
> > hooks are called after lxc_cmd_start and seccomp have been
> > initialised.
> > I am expecting the pre-start hooks to be executed in a pristine host
> > environment with some LXC variables. Is that wrong, an is it actually
> > run in a (more) permissive guest jail?
> > 
> > Is that a bug, or do I have wrong expectations about pre-start hooks?
> > 
> > I would be glad to help either way - I'd love to follow that rabbit
> > hole since it's been a great learning experience for me so far, or I
> > can add to the docs to prevent other people to fall into the same trap
> > :)
> > 
> > - Chris
> 
> I think I tracked down your problem:
> Failed to open connection to system bus: An AppArmor policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_call", sender="(null)" (inactive) interface="org.freedesktop.DBus" member="Hello" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)
> 
> Basically Apparmor is preventing lxc-start from talking to dbus. We'll
> have to change this for cgmanager anyway, so that should fix the problem
> for you at the same time.
> 
> In the meantime, you can simply add "dbus," to
> /etc/apparmor.d/abstractions/lxc/start-container and do
> "/etc/init.d/apparmor reload", that should fix it.

And added to the packaging bzr branch, the next PPA daily build will
have the change.

> 
> > 
> > 
> > [1]: Minimal pre-start hook: http://pastebin.ubuntu.com/6737406/
> > Resulting strace: http://pastebin.ubuntu.com/6737363/
> > 
> > On Sat, Jan 11, 2014 at 4:41 PM, Chris Glass <chris.glass at canonical.com> wrote:
> > > Hi,
> > >
> > > Unfortunately I don't think that's the cause of this particular issue.
> > > The error happens in *pre-start*, so the container is not yet started
> > > as far as I can tell (the documentation stipulates that it's executed
> > > in the host's environment).
> > > My changes do not need avahi to be installed on the guests at all. I
> > > want to query avahi from the host's environment for the info I need in
> > > pre-start, and put that in a file in the container's rootfs, then boot
> > > it normally.
> > >
> > > For the record I tried disabling rlimit_nproc in the host's avahi
> > > daemon (as well as setting it to a ridiculous number like 200), but
> > > that doesn't change anything.
> > >
> > > I'm not sure how to investigate this further... pointers welcome!
> > >
> > > - Chris
> > >
> > >
> > > On Fri, Jan 10, 2014 at 7:47 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> > >> Avahi sets its own rlimit_nproc to be exactly the # tasks it will spawn
> > >> as its userid.  If the avahi user in the container is the same uid #
> > >> as something in use on the host, then there may already be running
> > >> tasks and so rlimit_nproc is enforced and avahi can't run.
> > >>
> > >> If you use a user namespace this shouldn't be an issue (so long as
> > >> you don't share uid ranges between any containers).
> > >>
> > >> Otherwise, you have to change the avahi code to check whether it is in a
> > >> container before setting rlimit_nproc.  There is a patch to work around
> > >> this in the ubuntu trusty package, at least, if you wnat an example of
> > >> how to do it.
> > >>
> > >> -serge
> > >>
> > >> Quoting Chris Glass (chris.glass at canonical.com):
> > >>> Hi all,
> > >>>
> > >>> I'm currently hacking on a pre-start hook for the ubuntu and
> > >>> ubuntu-cloud templates that automatically makes ubuntu containers
> > >>> aware of squid-deb-proxy servers the host knows about.
> > >>>
> > >>> For this, I assume the squid-deb-proxy-client package is installed on
> > >>> the host. If you're not familiar with this package: it's basically a
> > >>> custom squid config for deb files along with an avahi config to expose
> > >>> an _apt_proxy._tcp service pointing to the squid.
> > >>> I currently run a squid-deb-proxy in a container.
> > >>>
> > >>> While running the avahi command on the host works fine (both as my
> > >>> user and as root), it fails when run from a pre-start script, and my
> > >>> knowledge of lxc is too limited to understand what could interfere
> > >>> with it.
> > >>>
> > >>> The failure:
> > >>> Running "avahi-browse -kprt _apt_proxy._tcp" in a lxc.hook.pre-start
> > >>> hook fails with "Failed to create client object: Access denied". It
> > >>> succeeds when ran at a normal shell on the host.
> > >>>
> > >>> A quick "whoami" in the same context yields "root", as expected, and
> > >>> switching to an unconfined apparmor profile does not change anything,
> > >>> so I suspect something more subtle is going on. How is the environment
> > >>> on "pre-start" hooks different?
> > >>>
> > >>> Could somebody shed some light here?
> > >>>
> > >>> Background:
> > >>> The reason for this is that I am a little frustrated to have to update
> > >>> my LXC container's apt proxy settings every time my squid-deb-proxy
> > >>> environment changes, and so this is an attempt at making this
> > >>> automatic. If you are curious and want to see it for yourself, have a
> > >>> look at [1], but please be aware that it's still work in progress.
> > >>>
> > >>> Thanks a lot for your help,
> > >>>
> > >>> - Chris
> > >>>
> > >>> links:
> > >>> --------
> > >>> [1]: https://github.com/chrisglass/lxc/blob/make-lxc-squid-deb-proxy-aware/hooks/squid-deb-proxy-client
> > >>> _______________________________________________
> > >>> lxc-devel mailing list
> > >>> lxc-devel at lists.linuxcontainers.org
> > >>> http://lists.linuxcontainers.org/listinfo/lxc-devel
> > >> _______________________________________________
> > >> lxc-devel mailing list
> > >> lxc-devel at lists.linuxcontainers.org
> > >> http://lists.linuxcontainers.org/listinfo/lxc-devel
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> 
> -- 
> Stéphane Graber
> Ubuntu developer
> http://www.ubuntu.com

> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140112/33f33487/attachment.pgp>