C11 threads

C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 1555 bytes --]

Hi everybody,
 after I have seen that gcc since 4.9 has a working implementation of
atomics, I convinced myself to implement C11 threads within musl.

I was really amazed, in essence this was just work of several
hours. The cleanness of musl made it really a pleasant experience.

I think the implementation is straight forward, there are only very
few minor glitches possible. But I thought that I first discuss with
you.

Choices :

 - all types are just the corresponding POSIX types

Issues :

 - a lot of C11 are just weak symbols. to avoid polluting the link
   space of a C11 application, I had to transform a lot of POSIX
   function symbols into weak symbols, too.

 - the include file threads.h includes pthread.h, so the namespace of
   the C11 implementation is poluted with the pthreads stuff

 - pthread_create and thrd_create are still too tightly linked
   together. It would be good to split the corresponding file in two,
   but this would make maintenance a bit more difficult.

 - there are still some symbols missing, for some of the functions
   that I implemented as macro. work for a winter day

 - I don't have much tests for all of this


I join my version of the threads.h file that has some comments on the
minor choices.

Jens


-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::




[-- Attachment #2: threads.h --]
[-- Type: text/x-chdr, Size: 3757 bytes --]

#ifndef _THREADS_H
#define _THREADS_H

#include <pthread.h>
#include <errno.h>

enum {
  thrd_success = 0,
  thrd_busy = EBUSY,
  thrd_error = EINVAL,
  thrd_nomem = ENOMEM,
  thrd_timedout = ETIMEDOUT,
};

enum {
  mtx_plain = 0,
  mtx_recursive = PTHREAD_MUTEX_RECURSIVE,
  mtx_check = PTHREAD_MUTEX_ERRORCHECK,
  // all mutexes are timed, here. so this is a no-op
  mtx_timed = 0,
};

_Static_assert((mtx_plain | mtx_recursive | mtx_check | mtx_timed) == 3,
               "This implementation only works on top of musl");

#define ONCE_FLAG_INIT PTHREAD_ONCE_INIT
#define TSS_DTOR_ITERATIONS PTHREAD_DESTRUCTOR_ITERATIONS

typedef pthread_cond_t cnd_t;
typedef pthread_mutex_t mtx_t;
typedef pthread_once_t once_flag;
typedef pthread_t thrd_t;
typedef pthread_key_t tss_t;

typedef int (*thrd_start_t)(void*);
typedef void (*tss_dtor_t)(void*);

#define thread_local _Thread_local

/* Don't create just additional symbols for the simple reason that the
   return type of the POSIX and C11 functions are different. */

int __pthread_once(once_flag *, void (*)(void));
// void call_once(once_flag *flag, void (*func)(void));
#define _call_once(FL, FU) (void)__pthread_once((FL), (FU))
#define call_once _call_once

_Noreturn void __pthread_exit(void *);
// _Noreturn void thrd_exit(int res);
#define _thrd_exit(RES) (__pthread_exit((void*)(intptr_t)(RES)))
#define thrd_exit _thrd_exit

int __pthread_cond_destroy(cnd_t *cond);
// void cnd_destroy(cnd_t *cond);
#define _cnd_destroy(CND) (void)__pthread_cond_destroy((CND))
#define cnd_destroy _cnd_destroy

void __pthread_mutex_destroy(mtx_t *mtx);
// void mtx_destroy(mtx_t *mtx);
#define _mtx_destroy(MTX) (void)__pthread_mutex_destroy((MTX))
#define mtx_destroy _mtx_destroy

int __pthread_mutex_unlock(mtx_t *mtx);
// int mtx_unlock(mtx_t *mtx);
#define _mtx_unlock(MTX) (__pthread_mutex_unlock((MTX)) ? thrd_error : thrd_success)
#define mtx_unlock _mtx_unlock

int __pthread_key_delete(tss_t key);
// void tss_delete(tss_t key);
#define _tss_delete(KEY)  (void)__pthread_key_delete((KEY))
#define tss_delete _tss_delete


/* Best of all worlds, these are just weak aliases */
int cnd_broadcast(cnd_t *);                   // pthread_cond_broadcast
int cnd_signal(cnd_t *);                      // pthread_cond_signal
int cnd_timedwait(cnd_t *restrict,            // pthread_cond_timedwait
                  mtx_t *restrict,
                  const struct timespec *restrict);
int cnd_wait(cnd_t *cond, mtx_t *mtx);        // pthread_cond_wait

int mtx_lock(mtx_t *);                        // pthread_mutex_lock
int mtx_timedlock(mtx_t *restrict,            // pthread_mutex_timedlock
                  const struct timespec *restrict);
int mtx_trylock(mtx_t *);                     // pthread_mutex_trylock

thrd_t thrd_current(void);                    // pthread_self
int thrd_equal(thrd_t, thrd_t);               // pthread_equal
int thrd_detach(thrd_t);                      // pthread_detach

int tss_create(tss_t *, tss_dtor_t);          // pthread_key_create
void *tss_get(tss_t);                         // pthread_getspecific
int tss_set(tss_t, void *);                   // pthread_setspecific


/* specialized implementations */

int cnd_init(cnd_t *);
int mtx_init(mtx_t *, int);

// reimplemented because of different calling convention
// shaves of a lot of code for attributes and cancelation
int thrd_create(thrd_t *,
                thrd_start_t,
                void *);

// reimplemented because of different calling convention
int thrd_join(thrd_t, int *);

// reimplemented because of different return value convention
int thrd_sleep(const struct timespec *,
               struct timespec *);

// reimplemented because of different return type
void thrd_yield(void);

#endif

Re: C11 threads

[Szabolcs Nagy]

* Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
>  after I have seen that gcc since 4.9 has a working implementation of
> atomics, I convinced myself to implement C11 threads within musl.
> 

the conclusion about threads.h was that it should be
abi compatible with glibc so we are waiting for them

so i think you should coordinate with libc-alpha too

> Choices :
> 
>  - all types are just the corresponding POSIX types
> 

if glibc does the same then it's ok

> Issues :
> 
>  - a lot of C11 are just weak symbols. to avoid polluting the link
>    space of a C11 application, I had to transform a lot of POSIX
>    function symbols into weak symbols, too.
> 

i think this not ok: posix usually requires inequal
function pointers for all public functions

>  - the include file threads.h includes pthread.h, so the namespace of
>    the C11 implementation is poluted with the pthreads stuff
> 

i think this is not ok in the final version

>  - pthread_create and thrd_create are still too tightly linked
>    together. It would be good to split the corresponding file in two,
>    but this would make maintenance a bit more difficult.
> 
>  - there are still some symbols missing, for some of the functions
>    that I implemented as macro. work for a winter day
> 

i assume you just have to add a ptototype before the macro
definition and provide a trivial wrapper implementation

>  - I don't have much tests for all of this

> 
> /* Don't create just additional symbols for the simple reason that the
>    return type of the POSIX and C11 functions are different. */
> 
> int __pthread_once(once_flag *, void (*)(void));
> // void call_once(once_flag *flag, void (*func)(void));
> #define _call_once(FL, FU) (void)__pthread_once((FL), (FU))
> #define call_once _call_once
> 

the additional symbols are necessary

and _[a-z] is not reserved for arbitrary usage for
the implementation (only for file scope identifiers)


> /* Best of all worlds, these are just weak aliases */
> int cnd_broadcast(cnd_t *);                   // pthread_cond_broadcast

posix is not yet aligned with c11, but i think they will
require inequal function pointers for these functions

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 3789 bytes --]

Hi,
thanks a lot for your feedback.

Am Freitag, den 25.07.2014, 12:40 +0200 schrieb Szabolcs Nagy:
> * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
> >  after I have seen that gcc since 4.9 has a working implementation of
> > atomics, I convinced myself to implement C11 threads within musl.
> > 
> 
> the conclusion about threads.h was that it should be
> abi compatible with glibc so we are waiting for them

They have that has a student summer project, so it seems to have low
priority for them.

> so i think you should coordinate with libc-alpha too
> 
> > Choices :
> > 
> >  - all types are just the corresponding POSIX types
> > 
> 
> if glibc does the same then it's ok

At least from the project description it seems that they prospect an
incremental implementation.

> > Issues :
> > 
> >  - a lot of C11 are just weak symbols. to avoid polluting the link
> >    space of a C11 application, I had to transform a lot of POSIX
> >    function symbols into weak symbols, too.
> > 

> i think this not ok: posix usually requires inequal
> function pointers for all public functions

C11 requires that the namespace not be polluted.

That conflict could be solved conveniently and easily by

 (1) compiling the same files twice and replacing the pthread symbol
     by the other in the second compilation

 (2) copy the .o file and to some ld magic

Which one to choose?

> >  - the include file threads.h includes pthread.h, so the namespace of
> >    the C11 implementation is poluted with the pthreads stuff
> > 
> 
> i think this is not ok in the final version

yes, I thought so, too. I would have to find a convenient way to just
include the magic constants. Or would it be ok, to just duplicate
them.

For the types this should be easy to have them typedef'ed to some
opaque struct.

> >  - pthread_create and thrd_create are still too tightly linked
> >    together. It would be good to split the corresponding file in two,
> >    but this would make maintenance a bit more difficult.
> > 
> >  - there are still some symbols missing, for some of the functions
> >    that I implemented as macro. work for a winter day
> > 
> 
> i assume you just have to add a ptototype before the macro
> definition and provide a trivial wrapper implementation

yes, was my thought, too.

> >  - I don't have much tests for all of this
> 
> > 
> > /* Don't create just additional symbols for the simple reason that the
> >    return type of the POSIX and C11 functions are different. */
> > 
> > int __pthread_once(once_flag *, void (*)(void));
> > // void call_once(once_flag *flag, void (*func)(void));
> > #define _call_once(FL, FU) (void)__pthread_once((FL), (FU))
> > #define call_once _call_once
> > 
> 
> the additional symbols are necessary
> 
> and _[a-z] is not reserved for arbitrary usage for
> the implementation (only for file scope identifiers)

Yes, I know, just was a temporary hack to get things going.

> > /* Best of all worlds, these are just weak aliases */
> > int cnd_broadcast(cnd_t *);                   // pthread_cond_broadcast
> 
> posix is not yet aligned with c11, but i think they will
> require inequal function pointers for these functions

The term "inequal function pointer" had nothing pop up. Could you give
a reference to the relevant part in POSIX that makes the requirement
for a "strong" symbol?

Thanks

Jens


-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::




[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Szabolcs Nagy]

* Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 13:06:44 +0200]:
> Am Freitag, den 25.07.2014, 12:40 +0200 schrieb Szabolcs Nagy:
> > * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
> > > /* Best of all worlds, these are just weak aliases */
> > > int cnd_broadcast(cnd_t *);                   // pthread_cond_broadcast
> > 
> > posix is not yet aligned with c11, but i think they will
> > require inequal function pointers for these functions
> 
> The term "inequal function pointer" had nothing pop up. Could you give
> a reference to the relevant part in POSIX that makes the requirement
> for a "strong" symbol?


hm i might be wrong:

there is a requirement in c and posix that the address of a
standard function can be taken and that == is only true for
function pointers if they point to the same function

but i don't see any requirement that each library function
must be distinct

and there is a dr that implies the opposite:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_078.html

without explanation the response says h can return 0.

the musl math library currently defines distinct long double
functions even if long double and double have the same
representation, they could be weak aliases if the standard
allows this..

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]

Am Freitag, den 25.07.2014, 15:04 +0200 schrieb Szabolcs Nagy:
> > The term "inequal function pointer" had nothing pop up. Could you give
> > a reference to the relevant part in POSIX that makes the requirement
> > for a "strong" symbol?
> 
> 
> hm i might be wrong:
> 
> there is a requirement in c and posix that the address of a
> standard function can be taken and that == is only true for
> function pointers if they point to the same function
> 
> but i don't see any requirement that each library function
> must be distinct
> 
> and there is a dr that implies the opposite:
> 
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_078.html
> 
> without explanation the response says h can return 0.

That DR is only for C. Would be interesting to have something for
POSIX, too.

> the musl math library currently defines distinct long double
> functions even if long double and double have the same
> representation, they could be weak aliases if the standard
> allows this..

It is completely sound as a policy to have no exported user functions
as weak aliases. It reduces the possibility the user shooting him or
herself in the foot.

Whatever choice would be fine with me.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::




[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Morten Welinder]

> the musl math library currently defines distinct long double [...]

You cannot compare a "double" function pointer to a "long double"
function pointer.  The types are incompatible, even if they have the
same representation.

Hence I don't believe a C program can even tell if they are the same.

M.





On Fri, Jul 25, 2014 at 9:04 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 13:06:44 +0200]:
>> Am Freitag, den 25.07.2014, 12:40 +0200 schrieb Szabolcs Nagy:
>> > * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
>> > > /* Best of all worlds, these are just weak aliases */
>> > > int cnd_broadcast(cnd_t *);                   // pthread_cond_broadcast
>> >
>> > posix is not yet aligned with c11, but i think they will
>> > require inequal function pointers for these functions
>>
>> The term "inequal function pointer" had nothing pop up. Could you give
>> a reference to the relevant part in POSIX that makes the requirement
>> for a "strong" symbol?
>
>
> hm i might be wrong:
>
> there is a requirement in c and posix that the address of a
> standard function can be taken and that == is only true for
> function pointers if they point to the same function
>
> but i don't see any requirement that each library function
> must be distinct
>
> and there is a dr that implies the opposite:
>
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_078.html
>
> without explanation the response says h can return 0.
>
> the musl math library currently defines distinct long double
> functions even if long double and double have the same
> representation, they could be weak aliases if the standard
> allows this..

Re: C11 threads

[Szabolcs Nagy]

* Morten Welinder <mwelinder@gmail.com> [2014-07-25 09:42:52 -0400]:
> > the musl math library currently defines distinct long double [...]
> 
> You cannot compare a "double" function pointer to a "long double"
> function pointer.  The types are incompatible, even if they have the
> same representation.
> 
> Hence I don't believe a C program can even tell if they are the same.
> 

conversion between incompatible function pointers is allowed

and i think == should work on converted pointers too:

void f(void) {}
void g(int x) {}
int h() {return f == (void(*)(void))g;}

Re: C11 threads

[Rich Felker]

On Fri, Jul 25, 2014 at 03:04:39PM +0200, Szabolcs Nagy wrote:
> > The term "inequal function pointer" had nothing pop up. Could you give
> > a reference to the relevant part in POSIX that makes the requirement
> > for a "strong" symbol?
> 
> 
> hm i might be wrong:
> 
> there is a requirement in c and posix that the address of a
> standard function can be taken and that == is only true for
> function pointers if they point to the same function
> 
> but i don't see any requirement that each library function
> must be distinct
> 
> and there is a dr that implies the opposite:
> 
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_078.html
> 
> without explanation the response says h can return 0.

Indeed there is no explanation. I don't think it could be related to
macros because my understanding is that only function-like macros are
allowed for the standard functions, i.e. you can override such macros
by using ().

Both C99 and C11 contain in 7.1.4 paragraph 1 a footnote (161 or 185,
respectively) reading:

"This means that an implementation shall provide an actual function
for each library function, even if it also provides a macro for that
function."

This suggests to me that there should be a one-to-one correspondence
between actual functions and library functions, but maybe I'm reading
too much into it.

> the musl math library currently defines distinct long double
> functions even if long double and double have the same
> representation, they could be weak aliases if the standard
> allows this..

Indeed. If this issue is resolved to allow it, I think we should make
this change and remove some otherwise-useless bloat.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 2576 bytes --]

Am Freitag, den 25.07.2014, 12:14 -0400 schrieb Rich Felker:
> On Fri, Jul 25, 2014 at 03:04:39PM +0200, Szabolcs Nagy wrote:
> > http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_078.html
> > 
> > without explanation the response says h can return 0.
> 
> Indeed there is no explanation. I don't think it could be related to
> macros because my understanding is that only function-like macros are
> allowed for the standard functions, i.e. you can override such macros
> by using ().

Indeed, I don't think that it has to do with macros per se, but with
the fact that one function can replace another one if the
implementation has the information that all requirements are
fulfilled. Indeed, if you have a version of memcpy that can cope with
overlapping memory blocks, you may also use it as memove.

> Both C99 and C11 contain in 7.1.4 paragraph 1 a footnote (161 or 185,
> respectively) reading:
> 
> "This means that an implementation shall provide an actual function
> for each library function, even if it also provides a macro for that
> function."
> 
> This suggests to me that there should be a one-to-one correspondence
> between actual functions and library functions, but maybe I'm reading
> too much into it.

I don't see where you get the one-to-one. For me this only says that
any library "function" can be used through the pointer-to-function.

I think the only intention is that any library function can be taken
as pointer, that pointer can be stored somewhere and then the
underlying function can be called from any context.

> > the musl math library currently defines distinct long double
> > functions even if long double and double have the same
> > representation, they could be weak aliases if the standard
> > allows this..
> 
> Indeed. If this issue is resolved to allow it, I think we should make
> this change and remove some otherwise-useless bloat.

If you are thinking in that direction, we definitively shouldn't use
trivial wrapper functions for the C11 functions. In all cases where
there is a trivial shift in the interface (such as the void return) I
would stay with the macro, and provide the additional symbol only for
the rare cases that someone asks for the address.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::





[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Rich Felker]

On Fri, Jul 25, 2014 at 11:59:06PM +0200, Jens Gustedt wrote:
> > > the musl math library currently defines distinct long double
> > > functions even if long double and double have the same
> > > representation, they could be weak aliases if the standard
> > > allows this..
> > 
> > Indeed. If this issue is resolved to allow it, I think we should make
> > this change and remove some otherwise-useless bloat.
> 
> If you are thinking in that direction, we definitively shouldn't use
> trivial wrapper functions for the C11 functions. In all cases where
> there is a trivial shift in the interface (such as the void return) I
> would stay with the macro, and provide the additional symbol only for
> the rare cases that someone asks for the address.

I don't want macros expanding to __-prefixed pthread names. This leaks
implementation internals into the ABI of binaries linked to musl and
makes it difficult or impossible to change those internals. glibc made
this choice in a lot of places (e.g. __strtol_internal, __ctype*, ...)
and it's one of the things that I (and many of our community members)
don't like about glibc.

Rich

Re: C11 threads

[Rich Felker]

On Fri, Jul 25, 2014 at 01:06:44PM +0200, Jens Gustedt wrote:
> Hi,
> thanks a lot for your feedback.
> 
> Am Freitag, den 25.07.2014, 12:40 +0200 schrieb Szabolcs Nagy:
> > * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
> > >  after I have seen that gcc since 4.9 has a working implementation of
> > > atomics, I convinced myself to implement C11 threads within musl.
> > > 
> > 
> > the conclusion about threads.h was that it should be
> > abi compatible with glibc so we are waiting for them
> 
> They have that has a student summer project, so it seems to have low
> priority for them.

Their GSOC funding for the project fell through due to missed
deadlines, so it's now an unfunded student project if it's even moving
forward at all. That may be the reason. :(

> > so i think you should coordinate with libc-alpha too
> > 
> > > Choices :
> > > 
> > >  - all types are just the corresponding POSIX types
> > > 
> > 
> > if glibc does the same then it's ok
> 
> At least from the project description it seems that they prospect an
> incremental implementation.

Yes, that was their decision and they were generally uninterested in
discussing whether it was the best decision. I tend to agree that it's
the right choice, but thought it could use more discussion before a
decision was made.

> > > Issues :
> > > 
> > >  - a lot of C11 are just weak symbols. to avoid polluting the link
> > >    space of a C11 application, I had to transform a lot of POSIX
> > >    function symbols into weak symbols, too.
> > > 
> 
> > i think this not ok: posix usually requires inequal
> > function pointers for all public functions
> 
> C11 requires that the namespace not be polluted.
> 
> That conflict could be solved conveniently and easily by
> 
>  (1) compiling the same files twice and replacing the pthread symbol
>      by the other in the second compilation
> 
>  (2) copy the .o file and to some ld magic
> 
> Which one to choose?

Neither of these is very reasonable. Probably the best is to
namespace-protect (i.e. prefix with __ and add the external name as a
weak alias) all of the pthread functions which need to be used for
implementing C11 functions, then add either second weak aliases (if
the equal function pointer issue is not a problem and the signature is
identical) or thin wrappers (probably my preference anyway) for the
C11 functions to call the __pthread ones.

The latter also allows us to add alternative implementations for the
C11 ones later on an incremental basis, if we want to. I generally
don't like the whole C11 threads mess, but the one potential advantage
they have is having weaker semantics which may allow better
performance, and definitely allows smaller size if the application
only uses them (but of course makes libc.a and libc.so larger).

The other thing nice about eventually having mostly-separate (I say
mostly because thread creation/destruction still needs to be unified,
I would think) code for POSIX and C11 would be that we could avoid
having permanent restrictions on the namespace usage for pthread
functions.

> > >  - the include file threads.h includes pthread.h, so the namespace of
> > >    the C11 implementation is poluted with the pthreads stuff
> > > 
> > 
> > i think this is not ok in the final version
> 
> yes, I thought so, too. I would have to find a convenient way to just
> include the magic constants. Or would it be ok, to just duplicate
> them.
> 
> For the types this should be easy to have them typedef'ed to some
> opaque struct.

For the types that vary per-arch, I don't see any way around having
them in the alltypes.h.in bits. As for the constants, which ones are
you talking about? I don't think it's so easy to directly use the
POSIX ones for C11 since they have some different semantics (flag bits
vs enumeration-style) for a few. Also this is one area where we still
need to be careful to match the glibc ABI.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 6091 bytes --]

Hello,

Am Freitag, den 25.07.2014, 11:41 -0400 schrieb Rich Felker:
> On Fri, Jul 25, 2014 at 01:06:44PM +0200, Jens Gustedt wrote:
> > Hi,
> > thanks a lot for your feedback.
> > 
> > Am Freitag, den 25.07.2014, 12:40 +0200 schrieb Szabolcs Nagy:
> > > * Jens Gustedt <jens.gustedt@inria.fr> [2014-07-25 12:00:37 +0200]:
> > > >  after I have seen that gcc since 4.9 has a working implementation of
> > > > atomics, I convinced myself to implement C11 threads within musl.
> > > > 
> > > 
> > > the conclusion about threads.h was that it should be
> > > abi compatible with glibc so we are waiting for them
> > 
> > They have that has a student summer project, so it seems to have low
> > priority for them.
> 
> Their GSOC funding for the project fell through due to missed
> deadlines, so it's now an unfunded student project if it's even moving
> forward at all. That may be the reason. :(
> 
> > > so i think you should coordinate with libc-alpha too
> > > 
> > > > Choices :
> > > > 
> > > >  - all types are just the corresponding POSIX types
> > > > 
> > > 
> > > if glibc does the same then it's ok
> > 
> > At least from the project description it seems that they prospect an
> > incremental implementation.
> 
> Yes, that was their decision and they were generally uninterested in
> discussing whether it was the best decision. I tend to agree that it's
> the right choice, but thought it could use more discussion before a
> decision was made.

Ideally, yes. For the time being we can mark this feature as
"experimental", I could even maintain this separately, if you
want. But honestly I don't expect much reaction from anywhere.

> > > > Issues :
> > > > 
> > > >  - a lot of C11 are just weak symbols. to avoid polluting the link
> > > >    space of a C11 application, I had to transform a lot of POSIX
> > > >    function symbols into weak symbols, too.
> > > > 
> > 
> > > i think this not ok: posix usually requires inequal
> > > function pointers for all public functions
> > 
> > C11 requires that the namespace not be polluted.
> > 
> > That conflict could be solved conveniently and easily by
> > 
> >  (1) compiling the same files twice and replacing the pthread symbol
> >      by the other in the second compilation
> > 
> >  (2) copy the .o file and to some ld magic
> > 
> > Which one to choose?
> 
> Neither of these is very reasonable. Probably the best is to
> namespace-protect (i.e. prefix with __ and add the external name as a
> weak alias) all of the pthread functions which need to be used for
> implementing C11 functions, then add either second weak aliases (if
> the equal function pointer issue is not a problem and the signature is
> identical) or thin wrappers (probably my preference anyway) for the
> C11 functions to call the __pthread ones.

Basically what I had from the start is the variant with two sets of
weak aliases.

> The latter also allows us to add alternative implementations for the
> C11 ones later on an incremental basis, if we want to.

I don't think that the weak alias approach inhibits this in any way.

> I generally
> don't like the whole C11 threads mess, but the one potential advantage
> they have is having weaker semantics which may allow better
> performance,

The weaker semantic is effectively too weak, and there are ongoing
efforts to amend that.

The advantages that I see that this doesn't need all that attr stuff
and has no concept of cancellation.

> and definitely allows smaller size if the application
> only uses them (but of course makes libc.a and libc.so larger).

If we use weak aliases, this basically blows up the symbol table a
bit.

When I strip my test executable, it is impressively small.

> The other thing nice about eventually having mostly-separate (I say
> mostly because thread creation/destruction still needs to be unified,
> I would think) code for POSIX and C11 would be that we could avoid
> having permanent restrictions on the namespace usage for pthread
> functions.
> 
> > > >  - the include file threads.h includes pthread.h, so the namespace of
> > > >    the C11 implementation is poluted with the pthreads stuff
> > > > 
> > > 
> > > i think this is not ok in the final version
> > 
> > yes, I thought so, too. I would have to find a convenient way to just
> > include the magic constants. Or would it be ok, to just duplicate
> > them.
> > 
> > For the types this should be easy to have them typedef'ed to some
> > opaque struct.
> 
> For the types that vary per-arch, I don't see any way around having
> them in the alltypes.h.in bits.

The only types that seem to be in bits are mutex and condition. All
others seem to be arch independent.

On the long we should probably have __pthread names in the alltypes.h
files and typedef these in pthread.h and threads.h respectively.

> As for the constants, which ones are
> you talking about?

the mtx_ constants are the critical ones.

thrd_error and friends can easily be mapped to EXXX error codes from
errno.h. Since these names are all reserved anyway, including errno.h
shouldn't do much harm.

> I don't think it's so easy to directly use the
> POSIX ones for C11 since they have some different semantics (flag bits
> vs enumeration-style) for a few.

the mtx_ ones are basically flags, just as the PTHREAD_MUTEX_ ones.
This fits well with the actual values of these in musl.

And then there is TSS_DTOR_ITERATIONS versus
PTHREAD_DESTRUCTOR_ITERATIONS, not a big deal. We just have to agree
on something.

> Also this is one area where we still
> need to be careful to match the glibc ABI.

Sure. In particular we shouldn't change any existing ABI.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::





[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Rich Felker]

On Fri, Jul 25, 2014 at 07:10:02PM +0200, Jens Gustedt wrote:
> > > > if glibc does the same then it's ok
> > > 
> > > At least from the project description it seems that they prospect an
> > > incremental implementation.
> > 
> > Yes, that was their decision and they were generally uninterested in
> > discussing whether it was the best decision. I tend to agree that it's
> > the right choice, but thought it could use more discussion before a
> > decision was made.
> 
> Ideally, yes. For the time being we can mark this feature as
> "experimental", I could even maintain this separately, if you
> want. But honestly I don't expect much reaction from anywhere.

Since glibc is going that way, I think we should keep the type sizes
the same but leave it explicitly undefined to mix calls to C11 and
POSIX functions on the same synchronization objects.

> > > C11 requires that the namespace not be polluted.
> > > 
> > > That conflict could be solved conveniently and easily by
> > > 
> > >  (1) compiling the same files twice and replacing the pthread symbol
> > >      by the other in the second compilation
> > > 
> > >  (2) copy the .o file and to some ld magic
> > > 
> > > Which one to choose?
> > 
> > Neither of these is very reasonable. Probably the best is to
> > namespace-protect (i.e. prefix with __ and add the external name as a
> > weak alias) all of the pthread functions which need to be used for
> > implementing C11 functions, then add either second weak aliases (if
> > the equal function pointer issue is not a problem and the signature is
> > identical) or thin wrappers (probably my preference anyway) for the
> > C11 functions to call the __pthread ones.
> 
> Basically what I had from the start is the variant with two sets of
> weak aliases.

OK. I didn't see that in your first email but perhaps I read it wrong.

> > The latter also allows us to add alternative implementations for the
> > C11 ones later on an incremental basis, if we want to.
> 
> I don't think that the weak alias approach inhibits this in any way.

It doesn't; I just meant that having the separate file/function
already in place makes the change smaller if we add an independent
implementation. But I agree it's a small difference and probably was
pointless to mention.

> > I generally
> > don't like the whole C11 threads mess, but the one potential advantage
> > they have is having weaker semantics which may allow better
> > performance,
> 
> The weaker semantic is effectively too weak, and there are ongoing
> efforts to amend that.

Could you elaborate? Perhaps we're thinking of different things.

One aspect is that all POSIX synchronization functions are full
barriers, whereas presumably the C11 ones are proper acquire/release
barriers as appropriate for the operation being performed.

Another is Austin Group issue #755, which I'm hoping the WG14 will not
rule the same way on. The Austin Group's resolution for POSIX mutexes
makes recursive and error-checking mutexes very expensive and requires
threads to maintain a record of all such mutexes they own. I have not
yet implemented this in musl but I plan to do so soon (BTW this needs
to be added to the roadmap).

> The advantages that I see that this doesn't need all that attr stuff
> and has no concept of cancellation.

Cancellation is rather irrelevant to the synchronization primitives.
Obviously C11's lack of cancellation makes C11 threads much easier to
implement if you're not also doing POSIX threads, but if you're doing
both, the fact that C11 lacks a cancellation function makes almost no
difference (just one simple cleanup function in condvar wait).

> > and definitely allows smaller size if the application
> > only uses them (but of course makes libc.a and libc.so larger).
> 
> If we use weak aliases, this basically blows up the symbol table a
> bit.
> 
> When I strip my test executable, it is impressively small.

Yes, this is one reason I don't want __-prefixed symbols for
everything, just functions that really _need_ namespace-safe versions.
We should probably do some linking tests against all the C11 functions
to make sure they're not pulling in functions outside the C11
namespace and add __-prefixed versions of those functions only, rather
than preemptively trying to guess what needs protection.

> > > For the types this should be easy to have them typedef'ed to some
> > > opaque struct.
> > 
> > For the types that vary per-arch, I don't see any way around having
> > them in the alltypes.h.in bits.
> 
> The only types that seem to be in bits are mutex and condition. All
> others seem to be arch independent.

Ah yes the others are probably not needed for C11.

> On the long we should probably have __pthread names in the alltypes.h
> files and typedef these in pthread.h and threads.h respectively.

That doesn't work because both sys/types.h and pthread.h have to
expose the pthread names.

> > As for the constants, which ones are
> > you talking about?
> 
> the mtx_ constants are the critical ones.
> 
> thrd_error and friends can easily be mapped to EXXX error codes from
> errno.h. Since these names are all reserved anyway, including errno.h
> shouldn't do much harm.

I don't think including errno.h implicitly is permitted. E* is only
reserved when errno.h is included. Anyway this is somewhere we wasnt
to agree with glibc on the values; I suspect they'll just use values
1,2,... rather than errno codes. If so, that means we really need a
wrapper function rather than just an alias.

> > I don't think it's so easy to directly use the
> > POSIX ones for C11 since they have some different semantics (flag bits
> > vs enumeration-style) for a few.
> 
> the mtx_ ones are basically flags, just as the PTHREAD_MUTEX_ ones.
> This fits well with the actual values of these in musl.

The PTHREAD_MUTEX_* ones are not flags; each corresponds to a single
type and they are not combinable. The C11 ones are combinable, but
mtx_timed is useless -- there's no need to care whether it will be
used for timed operations when it's initialized.

BTW n1570 only has 4 possible values to pass to mtx_init, but reads
(7.26.4.2 p2):

"The mtx_init function creates a mutex object with properties
indicated by type, which must have one of the six values:"

Any idea if six is just a mistake or if there are some values missing
from the list?

> And then there is TSS_DTOR_ITERATIONS versus
> PTHREAD_DESTRUCTOR_ITERATIONS, not a big deal. We just have to agree
> on something.
> 
> > Also this is one area where we still
> > need to be careful to match the glibc ABI.
> 
> Sure. In particular we shouldn't change any existing ABI.

Beyond not changing any existing ABI, I want to aim for making any new
ABIs added compatible.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 6749 bytes --]

Am Freitag, den 25.07.2014, 18:19 -0400 schrieb Rich Felker:
> Since glibc is going that way, I think we should keep the type sizes
> the same but leave it explicitly undefined to mix calls to C11 and
> POSIX functions on the same synchronization objects.

yes, defitively. Not even for the synchronization functions, basically
none from pthread and C11 thread should be mixed, I think. I thought
of figuring out a way to make this even a link error.

> Could you elaborate? Perhaps we're thinking of different things.

probably.

The C11 specification is so bad, that some of the functions don't even
have a semantic description of what they are supposed to achieve. E.g
tss_t completely lacks a description of what a destructor is, what
the "iterations" in TSS_DTOR_ITERATIONS describe etc. Parts of it are
or will be adressed TC, but there is much more. I wrote my findings up
here

http://gustedt.wordpress.com/2012/10/14/c11-defects-c-threads-are-not-realizable-with-posix-threads/

> One aspect is that all POSIX synchronization functions are full
> barriers, whereas presumably the C11 ones are proper acquire/release
> barriers as appropriate for the operation being performed.

That is probably the intention. But e.g mtx_lock is only required to
synchronize with calls to mtx_unlock, not with other calls to mtx_lock
or similar. The term "lock" isn't even defined.

> Another is Austin Group issue #755, which I'm hoping the WG14 will not
> rule the same way on. The Austin Group's resolution for POSIX mutexes
> makes recursive and error-checking mutexes very expensive and requires
> threads to maintain a record of all such mutexes they own. I have not
> yet implemented this in musl but I plan to do so soon (BTW this needs
> to be added to the roadmap).

I am not sure that I completely understand what this is about. But in
any case, C threads don't have error checking :)

> > The advantages that I see that this doesn't need all that attr stuff
> > and has no concept of cancellation.
> 
> Cancellation is rather irrelevant to the synchronization primitives.
> Obviously C11's lack of cancellation makes C11 threads much easier to
> implement if you're not also doing POSIX threads, but if you're doing
> both, the fact that C11 lacks a cancellation function makes almost no
> difference (just one simple cleanup function in condvar wait).

There is a bit more, maybe. __timedwait does cleanup push and pop and
messes around with cancelbuf. (But probably I just don't understand
enough what's going on, here.)

In any case, without attributes and possible cancelation, thrd_create
becomes significantly shorter than pthread_create.

And to my limited experience having well defined atomics that are
integrated in the language, often helps to completely avoid mutexes
and conditions.

> > > and definitely allows smaller size if the application
> > > only uses them (but of course makes libc.a and libc.so larger).
> > 
> > If we use weak aliases, this basically blows up the symbol table a
> > bit.
> > 
> > When I strip my test executable, it is impressively small.
> 
> Yes, this is one reason I don't want __-prefixed symbols for
> everything, just functions that really _need_ namespace-safe versions.
> We should probably do some linking tests against all the C11 functions
> to make sure they're not pulling in functions outside the C11
> namespace and add __-prefixed versions of those functions only, rather
> than preemptively trying to guess what needs protection.

I tried to do that already.

> > > > For the types this should be easy to have them typedef'ed to some
> > > > opaque struct.
> > > 
> > > For the types that vary per-arch, I don't see any way around having
> > > them in the alltypes.h.in bits.
> > 
> > The only types that seem to be in bits are mutex and condition. All
> > others seem to be arch independent.
> 
> Ah yes the others are probably not needed for C11.
> 
> > On the long we should probably have __pthread names in the alltypes.h
> > files and typedef these in pthread.h and threads.h respectively.
> 
> That doesn't work because both sys/types.h and pthread.h have to
> expose the pthread names.
> 
> > > As for the constants, which ones are
> > > you talking about?
> > 
> > the mtx_ constants are the critical ones.
> > 
> > thrd_error and friends can easily be mapped to EXXX error codes from
> > errno.h. Since these names are all reserved anyway, including errno.h
> > shouldn't do much harm.
> 
> I don't think including errno.h implicitly is permitted. E* is only
> reserved when errno.h is included. Anyway this is somewhere we wasnt
> to agree with glibc on the values; I suspect they'll just use values
> 1,2,... rather than errno codes. If so, that means we really need a
> wrapper function rather than just an alias.

Hm, not sure that I follow.

I only need EBUSY, EINVAL, ENOMEM, and ETIMEDOUT, and effectively only
that these are consistent with the rest of the C library, which for
this implementation of C threads will always be musl.

> > > I don't think it's so easy to directly use the
> > > POSIX ones for C11 since they have some different semantics (flag bits
> > > vs enumeration-style) for a few.
> > 
> > the mtx_ ones are basically flags, just as the PTHREAD_MUTEX_ ones.
> > This fits well with the actual values of these in musl.
> 
> The PTHREAD_MUTEX_* ones are not flags; each corresponds to a single
> type and they are not combinable.

ah, ok, anyhow the only one that I am really interested in is
PTHREAD_MUTEX_RECURSIVE. So perhaps I can push that down to the
implementation.

> The C11 ones are combinable, but mtx_timed is useless

yes, I think I already noted that in the file that I attached to my
initial mail

> -- there's no need to care whether it will be
> used for timed operations when it's initialized.
> 
> BTW n1570 only has 4 possible values to pass to mtx_init, but reads
> (7.26.4.2 p2):
> 
> "The mtx_init function creates a mutex object with properties
> indicated by type, which must have one of the six values:"
> 
> Any idea if six is just a mistake or if there are some values missing
> from the list?

It is just a leftover, and has/willbe been corrected.

To summarize, I'd need to get

EBUSY, EINVAL, ENOMEM, ETIMEDOUT and TSS_DTOR_ITERATIONS

from somewhere.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Rich Felker]

On Sat, Jul 26, 2014 at 01:26:09AM +0200, Jens Gustedt wrote:
> Am Freitag, den 25.07.2014, 18:19 -0400 schrieb Rich Felker:
> > Since glibc is going that way, I think we should keep the type sizes
> > the same but leave it explicitly undefined to mix calls to C11 and
> > POSIX functions on the same synchronization objects.
> 
> yes, defitively. Not even for the synchronization functions, basically
> none from pthread and C11 thread should be mixed, I think. I thought
> of figuring out a way to make this even a link error.

What do you mean? If you meant that calling C11 thread functions and
POSIX thread functions in the same program should be an error, I think
that's very wrong. The next issue of POSIX will be aligned with C11,
so both sets of interfaces will exist. Even if not for that, though,
it's just wrong conceptually to exclude the use of both. For example a
library written to ISO C could be using threads (C11 threads)
internally as an implementation detail, and this should not break a
caller which is using POSIX threads.

> > Could you elaborate? Perhaps we're thinking of different things.
> 
> probably.
> 
> The C11 specification is so bad, that some of the functions don't even
> have a semantic description of what they are supposed to achieve. E.g
> tss_t completely lacks a description of what a destructor is, what
> the "iterations" in TSS_DTOR_ITERATIONS describe etc. Parts of it are
> or will be adressed TC, but there is much more. I wrote my findings up
> here
> 
> http://gustedt.wordpress.com/2012/10/14/c11-defects-c-threads-are-not-realizable-with-posix-threads/

Thanks -- I'll check it out.

> > One aspect is that all POSIX synchronization functions are full
> > barriers, whereas presumably the C11 ones are proper acquire/release
> > barriers as appropriate for the operation being performed.
> 
> That is probably the intention. But e.g mtx_lock is only required to
> synchronize with calls to mtx_unlock, not with other calls to mtx_lock
> or similar. The term "lock" isn't even defined.

...

> > Another is Austin Group issue #755, which I'm hoping the WG14 will not
> > rule the same way on. The Austin Group's resolution for POSIX mutexes
> > makes recursive and error-checking mutexes very expensive and requires
> > threads to maintain a record of all such mutexes they own. I have not
> > yet implemented this in musl but I plan to do so soon (BTW this needs
> > to be added to the roadmap).
> 
> I am not sure that I completely understand what this is about. But in
> any case, C threads don't have error checking :)

It's about what happens when a thread exits whole holding a recursive
or errorchecking mutex. If the ownership of that mutex is tracked by a
thread id stored in the mutex (this is the only practical way to do
it), a newly created thread could wrongly become the owner of the
orphaned mutex just by getting the same thread id (by chance). The
only implementation options to avoid this are to have thread ids so
large that values never have to be reused, or to track the list of
mutexes owned by a thread so that it can change the owner to a dummy
value that will never match when it exits.

The obvious way to avoid this problem would be to add to the
specification:

"If a thread exits while it is the owner of a mutex, the behavior is
undefined."

Unfortunately the Austin Group did not want to do this. I'm hoping
someone can raise the issue with WG14 and that they'll decide
differently for C11 threads, so that C11 recursive mutexes can be more
efficient.

For POSIX mutexes, we're basically going to have to treat recursive
and errorchecking mutexes as robust mutexes to meet the requirements
of the standard, despite these requirements only affecting broken
programs that leave mutexes locked (and thus permanently locked) when
a thread exits.

> > > The advantages that I see that this doesn't need all that attr stuff
> > > and has no concept of cancellation.
> > 
> > Cancellation is rather irrelevant to the synchronization primitives.
> > Obviously C11's lack of cancellation makes C11 threads much easier to
> > implement if you're not also doing POSIX threads, but if you're doing
> > both, the fact that C11 lacks a cancellation function makes almost no
> > difference (just one simple cleanup function in condvar wait).
> 
> There is a bit more, maybe. __timedwait does cleanup push and pop and
> messes around with cancelbuf. (But probably I just don't understand
> enough what's going on, here.)

That's just to avoid having two different versions of __timedwait.
It's used for condvar wait, join, and a few other operations which are
cancellable. For mutexes the cancellation feature of __timedwait is
not used.

> In any case, without attributes and possible cancelation, thrd_create
> becomes significantly shorter than pthread_create.

The vast majority of the code in pthread_create is setting up the
stack, TLS, POSIX TSD, and the contents of the __pthread structure
(called TCB on other implementations). Attributes are a pretty small
part.

> And to my limited experience having well defined atomics that are
> integrated in the language, often helps to completely avoid mutexes
> and conditions.

I'm not sure about that. Atomics are mostly useful for the situations
where spinlocks would suffice. They don't help anywhere you would
expect a "wait" operation to happen (e.q. waiting for a queue to
become non-empty or non-full).

> > > > and definitely allows smaller size if the application
> > > > only uses them (but of course makes libc.a and libc.so larger).
> > > 
> > > If we use weak aliases, this basically blows up the symbol table a
> > > bit.
> > > 
> > > When I strip my test executable, it is impressively small.
> > 
> > Yes, this is one reason I don't want __-prefixed symbols for
> > everything, just functions that really _need_ namespace-safe versions.
> > We should probably do some linking tests against all the C11 functions
> > to make sure they're not pulling in functions outside the C11
> > namespace and add __-prefixed versions of those functions only, rather
> > than preemptively trying to guess what needs protection.
> 
> I tried to do that already.

OK.

> > > > > For the types this should be easy to have them typedef'ed to some
> > > > > opaque struct.
> > > > 
> > > > For the types that vary per-arch, I don't see any way around having
> > > > them in the alltypes.h.in bits.
> > > 
> > > The only types that seem to be in bits are mutex and condition. All
> > > others seem to be arch independent.
> > 
> > Ah yes the others are probably not needed for C11.
> > 
> > > On the long we should probably have __pthread names in the alltypes.h
> > > files and typedef these in pthread.h and threads.h respectively.
> > 
> > That doesn't work because both sys/types.h and pthread.h have to
> > expose the pthread names.
> > 
> > > > As for the constants, which ones are
> > > > you talking about?
> > > 
> > > the mtx_ constants are the critical ones.
> > > 
> > > thrd_error and friends can easily be mapped to EXXX error codes from
> > > errno.h. Since these names are all reserved anyway, including errno.h
> > > shouldn't do much harm.
> > 
> > I don't think including errno.h implicitly is permitted. E* is only
> > reserved when errno.h is included. Anyway this is somewhere we wasnt
> > to agree with glibc on the values; I suspect they'll just use values
> > 1,2,... rather than errno codes. If so, that means we really need a
> > wrapper function rather than just an alias.
> 
> Hm, not sure that I follow.
> 
> I only need EBUSY, EINVAL, ENOMEM, and ETIMEDOUT, and effectively only
> that these are consistent with the rest of the C library, which for
> this implementation of C threads will always be musl.

The point of ABI compatibility is that (at this point just some)
binaries and (more importantly) shared libraries without source that
were built/linked against glibc can be used with musl. But for this to
work, the values of the constants need to be the same.

> > > > I don't think it's so easy to directly use the
> > > > POSIX ones for C11 since they have some different semantics (flag bits
> > > > vs enumeration-style) for a few.
> > > 
> > > the mtx_ ones are basically flags, just as the PTHREAD_MUTEX_ ones.
> > > This fits well with the actual values of these in musl.
> > 
> > The PTHREAD_MUTEX_* ones are not flags; each corresponds to a single
> > type and they are not combinable.
> 
> ah, ok, anyhow the only one that I am really interested in is
> PTHREAD_MUTEX_RECURSIVE. So perhaps I can push that down to the
> implementation.

mtx_init needs to be a wrapper for pthread_mutex_init rather than an
alias anyway, since mtx_init takes an integer and pthread_mutex_init
takes a pointer to an attribute object.

> > The C11 ones are combinable, but mtx_timed is useless
> 
> yes, I think I already noted that in the file that I attached to my
> initial mail
> 
> > -- there's no need to care whether it will be
> > used for timed operations when it's initialized.
> > 
> > BTW n1570 only has 4 possible values to pass to mtx_init, but reads
> > (7.26.4.2 p2):
> > 
> > "The mtx_init function creates a mutex object with properties
> > indicated by type, which must have one of the six values:"
> > 
> > Any idea if six is just a mistake or if there are some values missing
> > from the list?
> 
> It is just a leftover, and has/willbe been corrected.
> 
> To summarize, I'd need to get
> 
> EBUSY, EINVAL, ENOMEM, ETIMEDOUT and TSS_DTOR_ITERATIONS

TSS_DTOR_ITERATIONS can just be defined to whatever the right value is
-- IIRC we use the minimum POSIX requires. It doesn't need to
magically sync with something else. If we ever need to change it we
can change both.

Obviously if the error values are used directly, duplicating them in
another header is more trouble since they vary per-arch. This is part
of why I would actually prefer not to use them for the thread function
result codes, but which we do will depend on which way glibc does it.
I can check in with them and see if they have a plan yet.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 5585 bytes --]

Am Freitag, den 25.07.2014, 22:24 -0400 schrieb Rich Felker:
> > yes, defitively. Not even for the synchronization functions, basically
> > none from pthread and C11 thread should be mixed, I think. I thought
> > of figuring out a way to make this even a link error.
> 
> What do you mean? If you meant that calling C11 thread functions and
> POSIX thread functions in the same program should be an error, I think
> that's very wrong.

Yes, I meant that.

> The next issue of POSIX will be aligned with C11,
> so both sets of interfaces will exist. Even if not for that, though,
> it's just wrong conceptually to exclude the use of both. For example a
> library written to ISO C could be using threads (C11 threads)
> internally as an implementation detail, and this should not break a
> caller which is using POSIX threads.

Ok, didn't occur to me. Thinking of it, even the C library is allowed
to use C threads to accelerate things, as long as it follows the as-if
rule.

> It's about what happens when a thread exits whole holding a recursive
> or errorchecking mutex. If the ownership of that mutex is tracked by a
> thread id stored in the mutex (this is the only practical way to do
> it), a newly created thread could wrongly become the owner of the
> orphaned mutex just by getting the same thread id (by chance). The
> only implementation options to avoid this are to have thread ids so
> large that values never have to be reused, or to track the list of
> mutexes owned by a thread so that it can change the owner to a dummy
> value that will never match when it exits.
> 
> The obvious way to avoid this problem would be to add to the
> specification:
> 
> "If a thread exits while it is the owner of a mutex, the behavior is
> undefined."

noted, I'll watch that something in that sense is included.

Probably this would need a bit more precision about "exits", I would
prefer to use "terminates". A thread terminates not necessarily
immediately after it returns or calls "thrd_exit", the tss destructors
should be called still from the same thread.

We should allow a thread to cleanup its mess with tss destructors.

> > And to my limited experience having well defined atomics that are
> > integrated in the language, often helps to completely avoid mutexes
> > and conditions.
> 
> I'm not sure about that. Atomics are mostly useful for the situations
> where spinlocks would suffice. They don't help anywhere you would
> expect a "wait" operation to happen (e.q. waiting for a queue to
> become non-empty or non-full).

Probably we have complementary experiences. Many uses of mutexes and
conditions in application code are about sharing and updating shared
resources. Often the resource protected is just a counter or other
small data. Especially counters are much better served with atomics.

And you are right mentioning that, in many situations spin locks do
effectively suffice. C11 has atomic_flag for that. Often locks are
just taken for critical sections of code that do a small task, just
some instructions.

> > I only need EBUSY, EINVAL, ENOMEM, and ETIMEDOUT, and effectively only
> > that these are consistent with the rest of the C library, which for
> > this implementation of C threads will always be musl.
> 
> The point of ABI compatibility is that (at this point just some)
> binaries and (more importantly) shared libraries without source that
> were built/linked against glibc can be used with musl. But for this to
> work, the values of the constants need to be the same.

yes, I am aware of that. That is why it is important to have the
thrd-constants and the E-constants in line.

The approach with weak aliases only works if the return codes of the
functions agree. We need

enum {
  thrd_success = 0,
  thrd_busy = EBUSY,
  thrd_error = EINVAL,
  thrd_nomem = ENOMEM,
  thrd_timedout = ETIMEDOUT,
};

and I don't think that there is much of a sensible way to do that
differently. Already the naming of the constants suggest that these
are the values that people (who?) had in mind when designing these
interfaces.

> mtx_init needs to be a wrapper for pthread_mutex_init rather than an
> alias anyway, since mtx_init takes an integer and pthread_mutex_init
> takes a pointer to an attribute object.

sure

> TSS_DTOR_ITERATIONS can just be defined to whatever the right value is
> -- IIRC we use the minimum POSIX requires. It doesn't need to
> magically sync with something else. If we ever need to change it we
> can change both.

ok

> Obviously if the error values are used directly, duplicating them in
> another header is more trouble since they vary per-arch. This is part
> of why I would actually prefer not to use them for the thread function
> result codes, but which we do will depend on which way glibc does it.
> I can check in with them and see if they have a plan yet.

yes, that would be good

I, on my site, will try to have something added to the C specification
that threads.h also includes errno.h. For the moment it is only
specified that it does so with time.h.

Perhaps it would even be good to have the thrd-constants also be
exported by errno.h. These are error codes, finally.


Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Rich Felker]

On Sat, Jul 26, 2014 at 09:16:40AM +0200, Jens Gustedt wrote:
> > It's about what happens when a thread exits whole holding a recursive
> > or errorchecking mutex. If the ownership of that mutex is tracked by a
> > thread id stored in the mutex (this is the only practical way to do
> > it), a newly created thread could wrongly become the owner of the
> > orphaned mutex just by getting the same thread id (by chance). The
> > only implementation options to avoid this are to have thread ids so
> > large that values never have to be reused, or to track the list of
> > mutexes owned by a thread so that it can change the owner to a dummy
> > value that will never match when it exits.
> > 
> > The obvious way to avoid this problem would be to add to the
> > specification:
> > 
> > "If a thread exits while it is the owner of a mutex, the behavior is
> > undefined."
> 
> noted, I'll watch that something in that sense is included.
> 
> Probably this would need a bit more precision about "exits", I would
> prefer to use "terminates". A thread terminates not necessarily
> immediately after it returns or calls "thrd_exit", the tss destructors
> should be called still from the same thread.
> 
> We should allow a thread to cleanup its mess with tss destructors.

Yes, I agree completely that "terminates" is the right wording.

> > > And to my limited experience having well defined atomics that are
> > > integrated in the language, often helps to completely avoid mutexes
> > > and conditions.
> > 
> > I'm not sure about that. Atomics are mostly useful for the situations
> > where spinlocks would suffice. They don't help anywhere you would
> > expect a "wait" operation to happen (e.q. waiting for a queue to
> > become non-empty or non-full).
> 
> Probably we have complementary experiences. Many uses of mutexes and
> conditions in application code are about sharing and updating shared
> resources. Often the resource protected is just a counter or other
> small data. Especially counters are much better served with atomics.
> 
> And you are right mentioning that, in many situations spin locks do
> effectively suffice. C11 has atomic_flag for that. Often locks are
> just taken for critical sections of code that do a small task, just
> some instructions.

I wasn't trying to say that spinlocks suffice in place of atomics, but
rather that atomics can rarely replace synchronization primitives, and
that in situations where you can't use a spinlock (because you expect
it to be waiting a long time for the lock, or because you want to do a
lot with the lock held) atomics are unlikely to solve the problem.

> > > I only need EBUSY, EINVAL, ENOMEM, and ETIMEDOUT, and effectively only
> > > that these are consistent with the rest of the C library, which for
> > > this implementation of C threads will always be musl.
> > 
> > The point of ABI compatibility is that (at this point just some)
> > binaries and (more importantly) shared libraries without source that
> > were built/linked against glibc can be used with musl. But for this to
> > work, the values of the constants need to be the same.
> 
> yes, I am aware of that. That is why it is important to have the
> thrd-constants and the E-constants in line.
> 
> The approach with weak aliases only works if the return codes of the
> functions agree. We need
> 
> enum {
>   thrd_success = 0,
>   thrd_busy = EBUSY,
>   thrd_error = EINVAL,
>   thrd_nomem = ENOMEM,
>   thrd_timedout = ETIMEDOUT,
> };
> 
> and I don't think that there is much of a sensible way to do that
> differently. Already the naming of the constants suggest that these
> are the values that people (who?) had in mind when designing these
> interfaces.

Actually, I disagree with your mappings. thrd_error is not going to
come from EINVAL (IIRC the only place POSIX specifies EINVAL anyway is
as an optional error when you already invoked UB) but rather from
attempting to lock a recursive mutex when the count is already maxed
out. And thrd_nomem is not going to come from ENOMEM, since
pthread_create returns EAGAIN, not ENOMEM, when memory for the thread
cannot be allocated.

With this in mind, your idea of using errno codes is seeming less and
less reasonable to me.

For reference, here's a summary of the only possible errors I see:

thrd_timedout - for timed operations

thrd_busy - for mtx_trylock

thrd_error - for mtx_lock or trylock (maxed recursive lock count), or
tss_create (no more tss slots)

thrd_nomem - for thrd_create

All of the other cases looked like things that cannot happen with musl
(or with any reasonable implementation).

> > Obviously if the error values are used directly, duplicating them in
> > another header is more trouble since they vary per-arch. This is part
> > of why I would actually prefer not to use them for the thread function
> > result codes, but which we do will depend on which way glibc does it.
> > I can check in with them and see if they have a plan yet.
> 
> yes, that would be good
> 
> I, on my site, will try to have something added to the C specification
> that threads.h also includes errno.h. For the moment it is only
> specified that it does so with time.h.
> 
> Perhaps it would even be good to have the thrd-constants also be
> exported by errno.h. These are error codes, finally.

I disagree here. There are many different kinds of result codes in C
and POSIX which are in different domains. If the thrd_* result codes
had been intended to be in the same domain as errno values, these
functions would have just been specified to return errno values. It's
possible that they might match errno values, but that's certainly an
implementation detail, not something that applications could
reasonably depend on, and therefore I think it makes no sense to
require or allow threads.h to expose errno.h.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 5557 bytes --]

Am Samstag, den 26.07.2014, 03:35 -0400 schrieb Rich Felker:
> On Sat, Jul 26, 2014 at 09:16:40AM +0200, Jens Gustedt wrote:
> > The approach with weak aliases only works if the return codes of the
> > functions agree. We need
> > 
> > enum {
> >   thrd_success = 0,
> >   thrd_busy = EBUSY,
> >   thrd_error = EINVAL,
> >   thrd_nomem = ENOMEM,
> >   thrd_timedout = ETIMEDOUT,
> > };
> > 
> > and I don't think that there is much of a sensible way to do that
> > differently. Already the naming of the constants suggest that these
> > are the values that people (who?) had in mind when designing these
> > interfaces.
> 
> Actually, I disagree with your mappings. thrd_error is not going to
> come from EINVAL (IIRC the only place POSIX specifies EINVAL anyway is
> as an optional error when you already invoked UB) but rather from
> attempting to lock a recursive mutex when the count is already maxed
> out.

I checked for all the functions that I marked for being suitable to be
aliased and that return int, and my impression was EINVAl was the only
error that could occur and that it was consistent with
thrd_error, and that other return values from POSIX functions such as
EDEADLK decribe behavior that is UB for C11.

But it seems I missed EAGAIN for recursive mutexes. I will reconsider.

> And thrd_nomem is not going to come from ENOMEM, since
> pthread_create returns EAGAIN, not ENOMEM, when memory for the thread
> cannot be allocated.

thrd_create can't be an alias, anyhow, so there is no problem in
translating return values.

(BTW, the C11 semantics for what thrd_nomem would mean are fuzzy
enought that returning thrd_error in all error cases would be
sufficient, I think)

Other functions that have return conventions that are not easily to
bring in line with POSIX are mtx_unlock and thrd_sleep, so they need
wrappers, too.


> With this in mind, your idea of using errno codes is seeming less and
> less reasonable to me.
> 
> For reference, here's a summary of the only possible errors I see:
> 
> thrd_timedout - for timed operations
> 
> thrd_busy - for mtx_trylock
> 
> thrd_error - for mtx_lock or trylock (maxed recursive lock count), or
> tss_create (no more tss slots)
> 
> thrd_nomem - for thrd_create
> 
> All of the other cases looked like things that cannot happen with musl
> (or with any reasonable implementation).
> 
> > > Obviously if the error values are used directly, duplicating them in
> > > another header is more trouble since they vary per-arch. This is part
> > > of why I would actually prefer not to use them for the thread function
> > > result codes, but which we do will depend on which way glibc does it.
> > > I can check in with them and see if they have a plan yet.
> > 
> > yes, that would be good
> > 
> > I, on my site, will try to have something added to the C specification
> > that threads.h also includes errno.h. For the moment it is only
> > specified that it does so with time.h.
> > 
> > Perhaps it would even be good to have the thrd-constants also be
> > exported by errno.h. These are error codes, finally.
> 
> I disagree here. There are many different kinds of result codes in C
> and POSIX which are in different domains. If the thrd_* result codes
> had been intended to be in the same domain as errno values, these
> functions would have just been specified to return errno values.

no.

Whereas you can normally suppose that additions to the C standard are
deeply discussed for all their implications with the existing parts,
historically this seems not to be the case for C11 threads.

The interface specification is just an artefact of being the one of a
proprietary library that existed at that time. That one had the
head-start of having two members of the committee being the owners of
the company that owns that library. (Probably these people had good
intentions, though. The discussion between the committee and the C
user community is simply not organized well.)

The starting point for C11 threads just was the reference manual of
that library, and the process of standardization only corrected the
most obvious incoherences. (E.g the fact that it used (but didn't
define) yet another type to specify times.)

Anybody with reason and with knowledge of POSIX and C (which are
standards from the same organization, ISO) would just have specified
error codes in line with pthreads. There are not many places in the
standard that uses enumeration constants for a similar purpose, and it
has not much of an advantage. There is the general requirement of the
standard that all constant integer expressions that it defines must be
testable by the preprocessor.

> It's
> possible that they might match errno values, but that's certainly an
> implementation detail, not something that applications could
> reasonably depend on, and therefore I think it makes no sense to
> require or allow threads.h to expose errno.h.

As you can see I strongly disagree here. We shouldn't introduce values
that semantically serve as error return values of library functions
and that are not regulated by errno.h and that aren't fixed in
compilation phase 3.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Rich Felker]

On Sat, Jul 26, 2014 at 10:32:55AM +0200, Jens Gustedt wrote:
> Anybody with reason and with knowledge of POSIX and C (which are
> standards from the same organization, ISO) would just have specified
> error codes in line with pthreads. There are not many places in the
> standard that uses enumeration constants for a similar purpose, and it
> has not much of an advantage. There is the general requirement of the
> standard that all constant integer expressions that it defines must be
> testable by the preprocessor.

I agree that they're ugly and contrary to style used elsewhere in the
standard, but I don't see how cross-polluting this mess with errno.h
makes it any less offensive.

> > It's
> > possible that they might match errno values, but that's certainly an
> > implementation detail, not something that applications could
> > reasonably depend on, and therefore I think it makes no sense to
> > require or allow threads.h to expose errno.h.
> 
> As you can see I strongly disagree here. We shouldn't introduce values
> that semantically serve as error return values of library functions
> and that are not regulated by errno.h and that aren't fixed in
> compilation phase 3.

Is this just ideological? POSIX has plenty of such values, such as the
GAI_*, REG_*, FNM_*, WRDE_*, etc. result codes. Maybe there are not
many (any?) in plain C, but C11 added them and I don't think there's
any use in trying to change it now...

Also C has no precedent for _returning_ error codes (excluding of
course Annex K); the errno.h constants are only used with the errno
object.

Rich

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 2723 bytes --]

Am Samstag, den 26.07.2014, 05:03 -0400 schrieb Rich Felker:
> On Sat, Jul 26, 2014 at 10:32:55AM +0200, Jens Gustedt wrote:
> > Anybody with reason and with knowledge of POSIX and C (which are
> > standards from the same organization, ISO) would just have specified
> > error codes in line with pthreads. There are not many places in the
> > standard that uses enumeration constants for a similar purpose, and it
> > has not much of an advantage. There is the general requirement of the
> > standard that all constant integer expressions that it defines must be
> > testable by the preprocessor.
> 
> I agree that they're ugly and contrary to style used elsewhere in the
> standard, but I don't see how cross-polluting this mess with errno.h
> makes it any less offensive.

It just makes it easier for implementations to chose a consistent set
of values.

> > As you can see I strongly disagree here. We shouldn't introduce values
> > that semantically serve as error return values of library functions
> > and that are not regulated by errno.h and that aren't fixed in
> > compilation phase 3.
> 
> Is this just ideological?

probably :)

> POSIX has plenty of such values, such as the
> GAI_*, REG_*, FNM_*, WRDE_*, etc. result codes. Maybe there are not
> many (any?) in plain C, but C11 added them and I don't think there's
> any use in trying to change it now...

POSIX seems to be a little less restrictive than C, here. It uses the
term "symbolic constant" where C always uses the term "macro"
consistenly all over the place, with threads.h and atomic.h the only
exceptions in using enumerations. And C has the explicit requirement
that all macros that are defined by the standard to resolve to an
"integer constant expression" be such that they can be used in
preprocessor directives.

> Also C has no precedent for _returning_ error codes (excluding of
> course Annex K); the errno.h constants are only used with the errno
> object.

Exactly, so they got it really bad by having added two optional parts
that don't agree on that handling, and that don't even aggree with
themselves. (thrd_sleep has yet another convention, -1 for interrupt
and other negative value for unspecified "failure", and timespec_get
gratuously uses the time base as a return value on success)

What a mess. I'll try to come up with a solution that is the least
intrusive.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: C11 threads

[Jens Gustedt]

[-- Attachment #1: Type: text/plain, Size: 1042 bytes --]

Am Samstag, den 26.07.2014, 09:16 +0200 schrieb Jens Gustedt:
> And you are right mentioning that, in many situations spin locks do
> effectively suffice. C11 has atomic_flag for that. Often locks are
> just taken for critical sections of code that do a small task, just
> some instructions.

Curiously, I just tracked down a use of pthread_mutex_lock and
pthread_mutex_unlock that I couldn't explain. It came from libatomic
as of gcc-4.9.0.

Looking into that, I see that they got the locking part for the non
lock-free atomics in their POSIX version plain wrong. Not only that
this sucks in the symbols which they shouldn't, but pthread_mutex_t is
just not the right tool for the task. It should be atomic_flag.

Jens

-- 
:: INRIA Nancy Grand Est ::: AlGorille ::: ICube/ICPS :::
:: ::::::::::::::: office Strasbourg : +33 368854536   ::
:: :::::::::::::::::::::: gsm France : +33 651400183   ::
:: ::::::::::::::: gsm international : +49 15737185122 ::
:: http://icube-icps.unistra.fr/index.php/Jens_Gustedt ::



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]