Re: [musl] ptc in pthread

[Rich Felker <dalias@libc.org>]

On Fri, Aug 16, 2024 at 10:51:53AM +0800, Zibin Liu wrote:
> Hi,
> 
> I’m not sure if this is the appropriate mailing list for my question. If
> it isn't, I’d appreciate it if someone could direct me to the correct
> one.
> 
> I’m currently studying pthreads and related concepts, and I’ve come
> across some code in pthread_create.c that I find a bit confusing.
> 
> In src/thread/pthread_create.c, I noticed the following:
> 
> int __pthread_create(pthread_t *restrict res, const pthread_attr_t
> *restrict attrp, void *(*entry)(void *), void *restrict arg)
> {
>     ......
> 
>     __acquire_ptc();
>     ......
>     __release_ptc();
>     ......
> fail:
>     __release_ptc();
>     return EAGAIN;
> }
> 
> It appears that when pthread_create is called, it acquires a lock
> (using __acquire_ptc()) and releases it afterward. I’m wondering why
> this locking mechanism is necessary.

See below.

> Additionally, I observed that a related lock is acquired during dlopen
> in ldso/dynlink.c:
> 
> void *dlopen(const char *file, int mode)
> {
>     ......
>     __inhibit_ptc();
>     ......
> end:
>     ......
>     __release_ptc();
>     ......
>     return p;
> }
> 
> From this, it seems that when dlopen is called, creating a new pthread
> is not allowed during the process. Does this mean that it’s entirely
> prohibited to create any threads (even if one were to use a custom thread
> library specifically within dlopen) during the execution of dlopen?

"Custom threads library" is completely outside the scope of musl, but
if you were doing that, you could not call any libc code from any
thread it created, since they would not be executing with the expected
state/context (of having a thread pointer pointing to a valid and
unique-to-the-thread libc thread structure that's a member of the libc
thread list).

It also would not be using the same lock, so nothing about this lock
would apply to it.

"During the execution of dlopen" is not a particularly interesting
limitation. I suspect you might be thinking that this includes
execution of ctors of loaded libraries, which might take arbitrarily
long in application-provided code, but it does not. All dlopen-held
locks are released once success of the dlopen is committed (i.e. once
dependency load and relocations have finished without errors).

> I also traced the commit logs and found that the 'ptc' mechanism was
> introduced in commit dcd6037, with the following message:
> 
> > I've re-appropriated the lock that was previously used for __synccall
> > (synchronizing set*id() syscalls between threads) as a general
> > pthread_create lock. it's a "backwards" rwlock where the "read"
> > operation is safe atomic modification of the live thread count, which
> > multiple threads can perform at the same time, and the "write"
> > operation is making sure the count does not increase during an
> > operation that depends on it remaining bounded (__synccall or dlopen).
> > in static-linked programs that don't use __synccall, this lock is a
> > no-op and has no cost.
> 
> Despite this, I’m still unclear on why dlopen needs to ensure that the
> thread count does not increase. Could someone provide more details on
> this?

The purpose of the lock has evolved considerably since it was
originally added, and I think it's not really well-named or
well-described as what it's really for.

The easiest way to see what it's actually doing though is to look at
what shared (global) state is accessed under the lock by
pthread_create:

- default attributes values: __default_stacksize and
  __default_guardsize

- TLS storage space requirement: libc.tls_size

In addition, there's the current thread count/thread list, accessed
and modified under __acquire_ptc lock from pthread_create, but it's
also protected by its own lock since 8f11e6127f. Previously, it was
written by pthread_create without further lock beyond the (shared,
read-mode __acquire_ptc lock), but that was fine since was only a
counter and the write was atomic.

So, in current usage, the ptc (pthread_create) lock is really behaving
as a simple rwlock for shared state that's read by pthread_create and
written to only rarely in a few other places.

Does this information help?

Rich