[lxc-devel] setns/fork(glibc) weird interaction, lxc-attach may hang sporadically
Serge Hallyn
serge.hallyn at ubuntu.com
Tue Apr 23 15:31:09 UTC 2013
Quoting Christian Seiler (christian at iwakd.de):
> Hi there,
>
> I've found a problem that comes from the interaction between setns()
> and glibc's implementation of fork() (at least on x86_64). It is rather
> complicated to describe, and forgive me if my mail is very long. I
> know of several ways how to address the problem but wanted to have a
> discussion first before actually changing something.
>
> For starters, let's read some glibc source code, just because it's
> so much fun. ;-)
thanks for the detailed explanation :) Fun indeed!
...
> But we don't know against what version of glibc lxc-attach is going to
> be build, so we need a better solution in the short term:
>
> - Call the fork syscall directly via syscall(__NR_fork)
> Since lxc-attach is single-threaded anyway, the things glibc does
lxc-attach isn't but at some point we'll want attach in the API, and
that is not single-threaded.
> aren't strictly needed, although the cache for getpid() is not
> updated afterwards.
>
> - Use clone(), where apparently, if I read the glibc source correctly,
> the wrapper doesn't use that logic. Makes life a bit more
> complicated because we need an extra function and can't just check
> the return value of a syscall. Advantages: glibc's clone() wrapper
> does seem to update the pid cache according to the source [2] and
> we could use CLONE_PARENT to keep only one process around (make the
> middle one exit immediately) and make the synchronization logic a
> bit simpler.
>
> - Do something else?
>
> The only possible complication could arrive from the fact that the
> pthread_atfork() handlers are not called with either the fork() syscall
> or the clone() function. This is relevant in cases where the host uses
> e.g. nss-ldap. That being said, we do exec() away pretty fast (and any
> open fds or such nss implementations will have O_CLOEXEC set), but we
> will do that only after invoking getpwuid() to get the shell if no
> command was specified. If a socket or the such is still open then, nss
> could produce some weird effects since it's the same socket as that of
> the parent process. On the other hand, lxc-attach doesn't do nss
> lookups at all before fork()ing away, so this is probably a non-issue.
>
> My suggestion would be to use clone() and to use that occasion to
> simplify the synchronization logic between all of those processes by
> using CLONE_PARENT. If I can get an agreement on this, I'll implement
> this, but I wanted to hear some thoughts in advance.
>
Agreed.
Thanks!
-serge
More information about the lxc-devel
mailing list