[lxc-devel] "pre-start" hooks and avahi

Sun Jan 12 17:12:08 UTC 2014

That seems to solve it indeed, thanks a lot!

As a followup question for my general knowledge - why did running the
container as "unconfined" not work? I assumed (naively) that it would
mean the container runs with no apparmor restrictions?

Thanks again, I'll share my changes on this mailing list once they are baked :)

- Chris

On Sun, Jan 12, 2014 at 5:41 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
> On Sun, Jan 12, 2014 at 11:39:15AM -0500, Stéphane Graber wrote:
>> On Sun, Jan 12, 2014 at 10:21:15AM +0100, Chris Glass wrote:
>> > Further investigation (with trace):
>> >
>> > It seems the failure I'm seeing is caused by not being able to talk to
>> > dbus's system bus in the pre-start hook [1].
>> > Looking at the code (src/lxc/start.c:355 ish), it seems pre-start
>> > hooks are called after lxc_cmd_start and seccomp have been
>> > initialised.
>> > I am expecting the pre-start hooks to be executed in a pristine host
>> > environment with some LXC variables. Is that wrong, an is it actually
>> > run in a (more) permissive guest jail?
>> >
>> > Is that a bug, or do I have wrong expectations about pre-start hooks?
>> >
>> > I would be glad to help either way - I'd love to follow that rabbit
>> > hole since it's been a great learning experience for me so far, or I
>> > can add to the docs to prevent other people to fall into the same trap
>> > :)
>> >
>> > - Chris
>>
>> I think I tracked down your problem:
>> Failed to open connection to system bus: An AppArmor policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_call", sender="(null)" (inactive) interface="org.freedesktop.DBus" member="Hello" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)
>>
>> Basically Apparmor is preventing lxc-start from talking to dbus. We'll
>> have to change this for cgmanager anyway, so that should fix the problem
>> for you at the same time.
>>
>> In the meantime, you can simply add "dbus," to
>> /etc/apparmor.d/abstractions/lxc/start-container and do
>> "/etc/init.d/apparmor reload", that should fix it.
>
> And added to the packaging bzr branch, the next PPA daily build will
> have the change.
>
>>
>> >
>> >
>> > [1]: Minimal pre-start hook: http://pastebin.ubuntu.com/6737406/
>> > Resulting strace: http://pastebin.ubuntu.com/6737363/
>> >
>> > On Sat, Jan 11, 2014 at 4:41 PM, Chris Glass <chris.glass at canonical.com> wrote:
>> > > Hi,
>> > >
>> > > Unfortunately I don't think that's the cause of this particular issue.
>> > > The error happens in *pre-start*, so the container is not yet started
>> > > as far as I can tell (the documentation stipulates that it's executed
>> > > in the host's environment).
>> > > My changes do not need avahi to be installed on the guests at all. I
>> > > want to query avahi from the host's environment for the info I need in
>> > > pre-start, and put that in a file in the container's rootfs, then boot
>> > > it normally.
>> > >
>> > > For the record I tried disabling rlimit_nproc in the host's avahi
>> > > daemon (as well as setting it to a ridiculous number like 200), but
>> > > that doesn't change anything.
>> > >
>> > > I'm not sure how to investigate this further... pointers welcome!
>> > >
>> > > - Chris
>> > >
>> > >
>> > > On Fri, Jan 10, 2014 at 7:47 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
>> > >> Avahi sets its own rlimit_nproc to be exactly the # tasks it will spawn
>> > >> as its userid.  If the avahi user in the container is the same uid #
>> > >> as something in use on the host, then there may already be running
>> > >> tasks and so rlimit_nproc is enforced and avahi can't run.
>> > >>
>> > >> If you use a user namespace this shouldn't be an issue (so long as
>> > >> you don't share uid ranges between any containers).
>> > >>
>> > >> Otherwise, you have to change the avahi code to check whether it is in a
>> > >> container before setting rlimit_nproc.  There is a patch to work around
>> > >> this in the ubuntu trusty package, at least, if you wnat an example of
>> > >> how to do it.
>> > >>
>> > >> -serge
>> > >>
>> > >> Quoting Chris Glass (chris.glass at canonical.com):
>> > >>> Hi all,
>> > >>>
>> > >>> I'm currently hacking on a pre-start hook for the ubuntu and
>> > >>> ubuntu-cloud templates that automatically makes ubuntu containers
>> > >>> aware of squid-deb-proxy servers the host knows about.
>> > >>>
>> > >>> For this, I assume the squid-deb-proxy-client package is installed on
>> > >>> the host. If you're not familiar with this package: it's basically a
>> > >>> custom squid config for deb files along with an avahi config to expose
>> > >>> an _apt_proxy._tcp service pointing to the squid.
>> > >>> I currently run a squid-deb-proxy in a container.
>> > >>>
>> > >>> While running the avahi command on the host works fine (both as my
>> > >>> user and as root), it fails when run from a pre-start script, and my
>> > >>> knowledge of lxc is too limited to understand what could interfere
>> > >>> with it.
>> > >>>
>> > >>> The failure:
>> > >>> Running "avahi-browse -kprt _apt_proxy._tcp" in a lxc.hook.pre-start
>> > >>> hook fails with "Failed to create client object: Access denied". It
>> > >>> succeeds when ran at a normal shell on the host.
>> > >>>
>> > >>> A quick "whoami" in the same context yields "root", as expected, and
>> > >>> switching to an unconfined apparmor profile does not change anything,
>> > >>> so I suspect something more subtle is going on. How is the environment
>> > >>> on "pre-start" hooks different?
>> > >>>
>> > >>> Could somebody shed some light here?
>> > >>>
>> > >>> Background:
>> > >>> The reason for this is that I am a little frustrated to have to update
>> > >>> my LXC container's apt proxy settings every time my squid-deb-proxy
>> > >>> environment changes, and so this is an attempt at making this
>> > >>> automatic. If you are curious and want to see it for yourself, have a
>> > >>> look at [1], but please be aware that it's still work in progress.
>> > >>>
>> > >>> Thanks a lot for your help,
>> > >>>
>> > >>> - Chris
>> > >>>
>> > >>> links:
>> > >>> --------
>> > >>> [1]: https://github.com/chrisglass/lxc/blob/make-lxc-squid-deb-proxy-aware/hooks/squid-deb-proxy-client
>> > >>> _______________________________________________
>> > >>> lxc-devel mailing list
>> > >>> lxc-devel at lists.linuxcontainers.org
>> > >>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>> > >> _______________________________________________
>> > >> lxc-devel mailing list
>> > >> lxc-devel at lists.linuxcontainers.org
>> > >> http://lists.linuxcontainers.org/listinfo/lxc-devel
>> > _______________________________________________
>> > lxc-devel mailing list
>> > lxc-devel at lists.linuxcontainers.org
>> > http://lists.linuxcontainers.org/listinfo/lxc-devel
>>
>> --
>> Stéphane Graber
>> Ubuntu developer
>> http://www.ubuntu.com
>
>
>
>> _______________________________________________
>> lxc-devel mailing list
>> lxc-devel at lists.linuxcontainers.org
>> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
>
> --
> Stéphane Graber
> Ubuntu developer
> http://www.ubuntu.com
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>