[lxc-devel] "pre-start" hooks and avahi

Sun Jan 12 17:19:55 UTC 2014

Hey,

"unconfined" only applies to the container itself, not to lxc-start
which has its own profile that always gets applied.

The first few hooks run in the host namespace so are constrained by the
lxc-start profile and not by the container's profile.

On Sun, Jan 12, 2014 at 06:12:08PM +0100, Chris Glass wrote:
> That seems to solve it indeed, thanks a lot!
> 
> As a followup question for my general knowledge - why did running the
> container as "unconfined" not work? I assumed (naively) that it would
> mean the container runs with no apparmor restrictions?
> 
> Thanks again, I'll share my changes on this mailing list once they are baked :)
> 
> - Chris
> 
> 
> On Sun, Jan 12, 2014 at 5:41 PM, Stéphane Graber <stgraber at ubuntu.com> wrote:
> > On Sun, Jan 12, 2014 at 11:39:15AM -0500, Stéphane Graber wrote:
> >> On Sun, Jan 12, 2014 at 10:21:15AM +0100, Chris Glass wrote:
> >> > Further investigation (with trace):
> >> >
> >> > It seems the failure I'm seeing is caused by not being able to talk to
> >> > dbus's system bus in the pre-start hook [1].
> >> > Looking at the code (src/lxc/start.c:355 ish), it seems pre-start
> >> > hooks are called after lxc_cmd_start and seccomp have been
> >> > initialised.
> >> > I am expecting the pre-start hooks to be executed in a pristine host
> >> > environment with some LXC variables. Is that wrong, an is it actually
> >> > run in a (more) permissive guest jail?
> >> >
> >> > Is that a bug, or do I have wrong expectations about pre-start hooks?
> >> >
> >> > I would be glad to help either way - I'd love to follow that rabbit
> >> > hole since it's been a great learning experience for me so far, or I
> >> > can add to the docs to prevent other people to fall into the same trap
> >> > :)
> >> >
> >> > - Chris
> >>
> >> I think I tracked down your problem:
> >> Failed to open connection to system bus: An AppArmor policy prevents this sender from sending this message to this recipient, 0 matched rules; type="method_call", sender="(null)" (inactive) interface="org.freedesktop.DBus" member="Hello" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus)
> >>
> >> Basically Apparmor is preventing lxc-start from talking to dbus. We'll
> >> have to change this for cgmanager anyway, so that should fix the problem
> >> for you at the same time.
> >>
> >> In the meantime, you can simply add "dbus," to
> >> /etc/apparmor.d/abstractions/lxc/start-container and do
> >> "/etc/init.d/apparmor reload", that should fix it.
> >
> > And added to the packaging bzr branch, the next PPA daily build will
> > have the change.
> >
> >>
> >> >
> >> >
> >> > [1]: Minimal pre-start hook: http://pastebin.ubuntu.com/6737406/
> >> > Resulting strace: http://pastebin.ubuntu.com/6737363/
> >> >
> >> > On Sat, Jan 11, 2014 at 4:41 PM, Chris Glass <chris.glass at canonical.com> wrote:
> >> > > Hi,
> >> > >
> >> > > Unfortunately I don't think that's the cause of this particular issue.
> >> > > The error happens in *pre-start*, so the container is not yet started
> >> > > as far as I can tell (the documentation stipulates that it's executed
> >> > > in the host's environment).
> >> > > My changes do not need avahi to be installed on the guests at all. I
> >> > > want to query avahi from the host's environment for the info I need in
> >> > > pre-start, and put that in a file in the container's rootfs, then boot
> >> > > it normally.
> >> > >
> >> > > For the record I tried disabling rlimit_nproc in the host's avahi
> >> > > daemon (as well as setting it to a ridiculous number like 200), but
> >> > > that doesn't change anything.
> >> > >
> >> > > I'm not sure how to investigate this further... pointers welcome!
> >> > >
> >> > > - Chris
> >> > >
> >> > >
> >> > > On Fri, Jan 10, 2014 at 7:47 PM, Serge Hallyn <serge.hallyn at ubuntu.com> wrote:
> >> > >> Avahi sets its own rlimit_nproc to be exactly the # tasks it will spawn
> >> > >> as its userid.  If the avahi user in the container is the same uid #
> >> > >> as something in use on the host, then there may already be running
> >> > >> tasks and so rlimit_nproc is enforced and avahi can't run.
> >> > >>
> >> > >> If you use a user namespace this shouldn't be an issue (so long as
> >> > >> you don't share uid ranges between any containers).
> >> > >>
> >> > >> Otherwise, you have to change the avahi code to check whether it is in a
> >> > >> container before setting rlimit_nproc.  There is a patch to work around
> >> > >> this in the ubuntu trusty package, at least, if you wnat an example of
> >> > >> how to do it.
> >> > >>
> >> > >> -serge
> >> > >>
> >> > >> Quoting Chris Glass (chris.glass at canonical.com):
> >> > >>> Hi all,
> >> > >>>
> >> > >>> I'm currently hacking on a pre-start hook for the ubuntu and
> >> > >>> ubuntu-cloud templates that automatically makes ubuntu containers
> >> > >>> aware of squid-deb-proxy servers the host knows about.
> >> > >>>
> >> > >>> For this, I assume the squid-deb-proxy-client package is installed on
> >> > >>> the host. If you're not familiar with this package: it's basically a
> >> > >>> custom squid config for deb files along with an avahi config to expose
> >> > >>> an _apt_proxy._tcp service pointing to the squid.
> >> > >>> I currently run a squid-deb-proxy in a container.
> >> > >>>
> >> > >>> While running the avahi command on the host works fine (both as my
> >> > >>> user and as root), it fails when run from a pre-start script, and my
> >> > >>> knowledge of lxc is too limited to understand what could interfere
> >> > >>> with it.
> >> > >>>
> >> > >>> The failure:
> >> > >>> Running "avahi-browse -kprt _apt_proxy._tcp" in a lxc.hook.pre-start
> >> > >>> hook fails with "Failed to create client object: Access denied". It
> >> > >>> succeeds when ran at a normal shell on the host.
> >> > >>>
> >> > >>> A quick "whoami" in the same context yields "root", as expected, and
> >> > >>> switching to an unconfined apparmor profile does not change anything,
> >> > >>> so I suspect something more subtle is going on. How is the environment
> >> > >>> on "pre-start" hooks different?
> >> > >>>
> >> > >>> Could somebody shed some light here?
> >> > >>>
> >> > >>> Background:
> >> > >>> The reason for this is that I am a little frustrated to have to update
> >> > >>> my LXC container's apt proxy settings every time my squid-deb-proxy
> >> > >>> environment changes, and so this is an attempt at making this
> >> > >>> automatic. If you are curious and want to see it for yourself, have a
> >> > >>> look at [1], but please be aware that it's still work in progress.
> >> > >>>
> >> > >>> Thanks a lot for your help,
> >> > >>>
> >> > >>> - Chris
> >> > >>>
> >> > >>> links:
> >> > >>> --------
> >> > >>> [1]: https://github.com/chrisglass/lxc/blob/make-lxc-squid-deb-proxy-aware/hooks/squid-deb-proxy-client
> >> > >>> _______________________________________________
> >> > >>> lxc-devel mailing list
> >> > >>> lxc-devel at lists.linuxcontainers.org
> >> > >>> http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> > >> _______________________________________________
> >> > >> lxc-devel mailing list
> >> > >> lxc-devel at lists.linuxcontainers.org
> >> > >> http://lists.linuxcontainers.org/listinfo/lxc-devel
> >> > _______________________________________________
> >> > lxc-devel mailing list
> >> > lxc-devel at lists.linuxcontainers.org
> >> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >>
> >> --
> >> Stéphane Graber
> >> Ubuntu developer
> >> http://www.ubuntu.com
> >
> >
> >
> >> _______________________________________________
> >> lxc-devel mailing list
> >> lxc-devel at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-devel
> >
> >
> > --
> > Stéphane Graber
> > Ubuntu developer
> > http://www.ubuntu.com
> >
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140112/543291e8/attachment.pgp>