[lxc-devel] Working with glibc (PID/TID caches).

[Eric W. Biederman]

Serge Hallyn <serge.hallyn at ubuntu.com> writes:

> Quoting Eric W. Biederman (ebiederm at xmission.com):
>> It would be nice if at least malloc and C++ new were safe (and
>> documented as safe) after fork in a pthread environment.  That would go
>> a long ways to allowing running interesting set up code without having
>> to jump through hoops.
>
> I'm pretty sure malloc is in fact thread-safe.  For some time I was
> paranoid about file table operations (like fopen) and mutexed them
> all, but we eventually realized that those are also thread-safe.

The dangerous part I was referring to is what happens when someone
holds a mutex and calls fork(3) or clone(3).

I had missed the existence of pthread_atfork and malloc in glibc
very clearly has atfork handlers so malloc should be safe after
fork(3).

However I justed tested it and clone(3) without CLONE_VM does not
call the atfork handlers which is a nasty problem to work with.

***** Can libc please be modified to call the atfork handlers from
      clone(3) without CLONE_VM.

      Pretty Please with sugar on top.

> The only hoop we run through in lxc that I can think of is that we
> use pthread_atfork() to drop any mutexes which some thread may have
> been holding at fork time.  With that, we have Çaglar doing some
> rather impressive concurrency runs using the lxc-go api, and afaik
> no issues currently (except in libnih/libdbus, which we have to
> paper over by mutexing it, sadly).

I had failed to notice the pthread_atfork handlers and those definitely
solve (or at least make it possible to solve) the nasties that pthreads
introduce into fork.

If we can get those same fixes for clone(3) that would be a real help.

Serge are you using unshare in liblxc right now?  Using clone(3) where
it doesn't call the pthread_atfork handlers is starting to scare me a
little.


Eric