FreeBSD's Google Summer of Code 2016

FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

Hello;

Just thought I'd point this out in case there are students looking
for a summer project (paid by Google):

In FreeBSD we are always open to alternatives. For this GSoC we would 
consider someone willing to work on a musl port.

https://wiki.freebsd.org/SummerOfCodeIdeas#Port_musl_libc

For such a port you obviously have to have love for the C language
and specific skills (version control, knowledge of the build system, 
UNIX/syscall knowledge), but I think it may very well be a fun
project for the right person.

We generally ask for very detailed proposals and it certainly
would help if you get to try FreeBSD before considering a
proposal, The process is competitive so we are interested
in your background.

We are aware musl is centered around linux but just as someone
managed to port glibc to FreeBSD we think it should be possible
to make a port clean port of musl. This project would only make
sense if the code is upstreamed, so at some stage we would also
appreciate support from musl developers.

For applying you should follow the standard procedure with Google.
https://summerofcode.withgoogle.com/how-it-works/

Best regards,

Pedro.

Re: FreeBSD's Google Summer of Code 2016

[Rich Felker]

On Sat, Mar 05, 2016 at 03:11:28PM -0500, Pedro Giffuni wrote:
> Hello;
> 
> Just thought I'd point this out in case there are students looking
> for a summer project (paid by Google):
> 
> In FreeBSD we are always open to alternatives. For this GSoC we
> would consider someone willing to work on a musl port.
> 
> https://wiki.freebsd.org/SummerOfCodeIdeas#Port_musl_libc
> 
> For such a port you obviously have to have love for the C language
> and specific skills (version control, knowledge of the build system,
> UNIX/syscall knowledge), but I think it may very well be a fun
> project for the right person.
> 
> We generally ask for very detailed proposals and it certainly
> would help if you get to try FreeBSD before considering a
> proposal, The process is competitive so we are interested
> in your background.
> 
> We are aware musl is centered around linux but just as someone
> managed to port glibc to FreeBSD we think it should be possible
> to make a port clean port of musl. This project would only make
> sense if the code is upstreamed, so at some stage we would also
> appreciate support from musl developers.
> 
> For applying you should follow the standard procedure with Google.
> https://summerofcode.withgoogle.com/how-it-works/

Nice!

From our side (musl), the biggest obstacle when people have looked at
doing such a port in the past seems to be missing syscalls, especially
futex and related functionality. For that reason, up til now,
FreeBSD's Linux syscall layer has seemed more functional, and a better
basis for musl-on-FreeBSD, than the native syscall layer. I think a
viable proposal needs a solution to this problem on the kernel side --
perhaps exposing some of these syscalls that are Linux-emulation-only
now in the native API?

Rich

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

Hello Rich;

On 03/05/16 16:25, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 03:11:28PM -0500, Pedro Giffuni wrote:
>> Hello;
>>
>> Just thought I'd point this out in case there are students looking
>> for a summer project (paid by Google):
>>
>> In FreeBSD we are always open to alternatives. For this GSoC we
>> would consider someone willing to work on a musl port.
>>
>> https://wiki.freebsd.org/SummerOfCodeIdeas#Port_musl_libc
>>
>> For such a port you obviously have to have love for the C language
>> and specific skills (version control, knowledge of the build system,
>> UNIX/syscall knowledge), but I think it may very well be a fun
>> project for the right person.
>>
>> We generally ask for very detailed proposals and it certainly
>> would help if you get to try FreeBSD before considering a
>> proposal, The process is competitive so we are interested
>> in your background.
>>
>> We are aware musl is centered around linux but just as someone
>> managed to port glibc to FreeBSD we think it should be possible
>> to make a port clean port of musl. This project would only make
>> sense if the code is upstreamed, so at some stage we would also
>> appreciate support from musl developers.
>>
>> For applying you should follow the standard procedure with Google.
>> https://summerofcode.withgoogle.com/how-it-works/
>
> Nice!
>
>  From our side (musl), the biggest obstacle when people have looked at
> doing such a port in the past seems to be missing syscalls, especially
> futex and related functionality. For that reason, up til now,
> FreeBSD's Linux syscall layer has seemed more functional, and a better
> basis for musl-on-FreeBSD, than the native syscall layer. I think a
> viable proposal needs a solution to this problem on the kernel side --
> perhaps exposing some of these syscalls that are Linux-emulation-only
> now in the native API?
>

First of all, great to hear there is interest on the musl side too.

I think the biggest precedent of porting linux-oriented C libraries
came from Debian's kFreeBSD. We accomodated a little by for them
by defining __FreeBSD_kernel__ in sys/param.h.

While using the optional linux-abi futex in FreeBSD could be an option,
it is not really the cleanest option. The Debian guys did a port of
NPTL using regular pthreads:

http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702

I am certain this will require more research but it would be useful
for other ports as well.

Pedro.

Re: FreeBSD's Google Summer of Code 2016

[Rich Felker]

On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
> First of all, great to hear there is interest on the musl side too.
> 
> I think the biggest precedent of porting linux-oriented C libraries
> came from Debian's kFreeBSD. We accomodated a little by for them
> by defining __FreeBSD_kernel__ in sys/param.h.
> 
> While using the optional linux-abi futex in FreeBSD could be an option,
> it is not really the cleanest option. The Debian guys did a port of
> NPTL using regular pthreads:
> 
> http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
> 
> I am certain this will require more research but it would be useful
> for other ports as well.

Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
the lll stuff) in their pthread primitives which makes this
"possible". I call it gratuitous because it's really really hard to
achieve correct implementations of the pthread sync primitives that
don't have serious corner-case bugs, and it's unlikely that their
abstractions actually suffice to make correct alternate
implementations.

musl does not have any such abstraction. We require a compare-and-swap
operation or equivalent on which arbitrary atomic operations can be
constructed, a futex or equivalent operation that's roughly
while(*addr==expected) sleep(), and implement all the sync primitives
just once on top of these.

In a worst case, futex can be emulated trivially by having wait and
wake both be nops; then everything just spins at 100% cpu load. Of
course this does not give very nice results. It's possible futex could
be emulated better; certainly it can be within a single process where
the futex wait queue could just be implemented in userspace, but
emulating process-shared futexes effectively this way sounds
nontrivial. Since the functionality is already there in the kernel,
though, just exposing it sounds like a much better solution to me.

Rich

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

On 03/05/16 18:32, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
>> First of all, great to hear there is interest on the musl side too.
>>
>> I think the biggest precedent of porting linux-oriented C libraries
>> came from Debian's kFreeBSD. We accomodated a little by for them
>> by defining __FreeBSD_kernel__ in sys/param.h.
>>
>> While using the optional linux-abi futex in FreeBSD could be an option,
>> it is not really the cleanest option. The Debian guys did a port of
>> NPTL using regular pthreads:
>>

Of course I ahould have meant "based on regular FreeBSD kernel services".

>> http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
>>
>> I am certain this will require more research but it would be useful
>> for other ports as well.
>

We could ask Petr Salinger for the details, but I am pretty sure
FreeBSD has the required functionality natively.

> Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
> the lll stuff) in their pthread primitives which makes this
> "possible". I call it gratuitous because it's really really hard to
> achieve correct implementations of the pthread sync primitives that
> don't have serious corner-case bugs, and it's unlikely that their
> abstractions actually suffice to make correct alternate
> implementations.
>
> musl does not have any such abstraction. We require a compare-and-swap
> operation or equivalent on which arbitrary atomic operations can be
> constructed, a futex or equivalent operation that's roughly
> while(*addr==expected) sleep(), and implement all the sync primitives
> just once on top of these.
>

I am not a threading expert (or even a CS guy), but it sounds like
mutex(9) with condvar(9) would do [1]:

https://www.freebsd.org/doc/en/books/arch-handbook/locking.html

Cheers,

Pedro.

[1] You can watch the manpages here:
https://www.freebsd.org/cgi/man.cgi

Re: FreeBSD's Google Summer of Code 2016

[Rich Felker]

On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
> 
> 
> On 03/05/16 18:32, Rich Felker wrote:
> >On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
> >>First of all, great to hear there is interest on the musl side too.
> >>
> >>I think the biggest precedent of porting linux-oriented C libraries
> >>came from Debian's kFreeBSD. We accomodated a little by for them
> >>by defining __FreeBSD_kernel__ in sys/param.h.
> >>
> >>While using the optional linux-abi futex in FreeBSD could be an option,
> >>it is not really the cleanest option. The Debian guys did a port of
> >>NPTL using regular pthreads:
> >>
> 
> Of course I ahould have meant "based on regular FreeBSD kernel services".
> 
> >>http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
> >>
> >>I am certain this will require more research but it would be useful
> >>for other ports as well.
> >
> 
> We could ask Petr Salinger for the details, but I am pretty sure
> FreeBSD has the required functionality natively.
> 
> >Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
> >the lll stuff) in their pthread primitives which makes this
> >"possible". I call it gratuitous because it's really really hard to
> >achieve correct implementations of the pthread sync primitives that
> >don't have serious corner-case bugs, and it's unlikely that their
> >abstractions actually suffice to make correct alternate
> >implementations.
> >
> >musl does not have any such abstraction. We require a compare-and-swap
> >operation or equivalent on which arbitrary atomic operations can be
> >constructed, a futex or equivalent operation that's roughly
> >while(*addr==expected) sleep(), and implement all the sync primitives
> >just once on top of these.
> >
> 
> I am not a threading expert (or even a CS guy), but it sounds like
> mutex(9) with condvar(9) would do [1]:

No, they don't satisfy the needs of musl; they have their own
additional storage requirements and are probably not AS-safe. It might
be possible to use them to implement a userspace-emulated futex queue
(only if they are AS-safe), but I don't see a way to extend that to
the process-shared case.

Rich

Re: FreeBSD's Google Summer of Code 2016

[Rich Felker]

On Sat, Mar 05, 2016 at 07:25:47PM -0500, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
> > 
> > 
> > On 03/05/16 18:32, Rich Felker wrote:
> > >On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
> > >>First of all, great to hear there is interest on the musl side too.
> > >>
> > >>I think the biggest precedent of porting linux-oriented C libraries
> > >>came from Debian's kFreeBSD. We accomodated a little by for them
> > >>by defining __FreeBSD_kernel__ in sys/param.h.
> > >>
> > >>While using the optional linux-abi futex in FreeBSD could be an option,
> > >>it is not really the cleanest option. The Debian guys did a port of
> > >>NPTL using regular pthreads:
> > >>
> > 
> > Of course I ahould have meant "based on regular FreeBSD kernel services".
> > 
> > >>http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
> > >>
> > >>I am certain this will require more research but it would be useful
> > >>for other ports as well.
> > >
> > 
> > We could ask Petr Salinger for the details, but I am pretty sure
> > FreeBSD has the required functionality natively.
> > 
> > >Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
> > >the lll stuff) in their pthread primitives which makes this
> > >"possible". I call it gratuitous because it's really really hard to
> > >achieve correct implementations of the pthread sync primitives that
> > >don't have serious corner-case bugs, and it's unlikely that their
> > >abstractions actually suffice to make correct alternate
> > >implementations.
> > >
> > >musl does not have any such abstraction. We require a compare-and-swap
> > >operation or equivalent on which arbitrary atomic operations can be
> > >constructed, a futex or equivalent operation that's roughly
> > >while(*addr==expected) sleep(), and implement all the sync primitives
> > >just once on top of these.
> > >
> > 
> > I am not a threading expert (or even a CS guy), but it sounds like
> > mutex(9) with condvar(9) would do [1]:
> 
> No, they don't satisfy the needs of musl; they have their own
> additional storage requirements and are probably not AS-safe. It might
> be possible to use them to implement a userspace-emulated futex queue
> (only if they are AS-safe), but I don't see a way to extend that to
> the process-shared case.

Actually these look like kernelspace APIs, not userspace, unless I'm
mistaken. In that case I don't see how they're relevant/usable. Is tht
correct?

Rich

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

On 03/05/16 19:25, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
>>
>>
>> On 03/05/16 18:32, Rich Felker wrote:
>>> On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
>>>> First of all, great to hear there is interest on the musl side too.
>>>>
>>>> I think the biggest precedent of porting linux-oriented C libraries
>>>> came from Debian's kFreeBSD. We accomodated a little by for them
>>>> by defining __FreeBSD_kernel__ in sys/param.h.
>>>>
>>>> While using the optional linux-abi futex in FreeBSD could be an option,
>>>> it is not really the cleanest option. The Debian guys did a port of
>>>> NPTL using regular pthreads:
>>>>
>>
>> Of course I ahould have meant "based on regular FreeBSD kernel services".
>>
>>>> http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
>>>>
>>>> I am certain this will require more research but it would be useful
>>>> for other ports as well.
>>>
>>
>> We could ask Petr Salinger for the details, but I am pretty sure
>> FreeBSD has the required functionality natively.
>>
>>> Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
>>> the lll stuff) in their pthread primitives which makes this
>>> "possible". I call it gratuitous because it's really really hard to
>>> achieve correct implementations of the pthread sync primitives that
>>> don't have serious corner-case bugs, and it's unlikely that their
>>> abstractions actually suffice to make correct alternate
>>> implementations.
>>>
>>> musl does not have any such abstraction. We require a compare-and-swap
>>> operation or equivalent on which arbitrary atomic operations can be
>>> constructed, a futex or equivalent operation that's roughly
>>> while(*addr==expected) sleep(), and implement all the sync primitives
>>> just once on top of these.
>>>
>>
>> I am not a threading expert (or even a CS guy), but it sounds like
>> mutex(9) with condvar(9) would do [1]:
>
> No, they don't satisfy the needs of musl; they have their own
> additional storage requirements and are probably not AS-safe. It might
> be possible to use them to implement a userspace-emulated futex queue
> (only if they are AS-safe), but I don't see a way to extend that to
> the process-shared case.
>

OK, it looks like sema(9) may be nearer (and also simpler but slower).
For the process-shared case libthr(2) uses the stuff in sys/utmx.h,
shich should be looked at but it is not documented[1].

Luckily Ed, the developer that would be mentoring the project, knows 
this stuff better than I do.

Pedro.

[1] http://fxr.watson.org/fxr/source/sys/umtx.h?v=FREEBSD8

Re: FreeBSD's Google Summer of Code 2016

[Rich Felker]

On Sat, Mar 05, 2016 at 07:59:35PM -0500, Pedro Giffuni wrote:
> 
> 
> On 03/05/16 19:25, Rich Felker wrote:
> >On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
> >>
> >>
> >>On 03/05/16 18:32, Rich Felker wrote:
> >>>On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
> >>>>First of all, great to hear there is interest on the musl side too.
> >>>>
> >>>>I think the biggest precedent of porting linux-oriented C libraries
> >>>>came from Debian's kFreeBSD. We accomodated a little by for them
> >>>>by defining __FreeBSD_kernel__ in sys/param.h.
> >>>>
> >>>>While using the optional linux-abi futex in FreeBSD could be an option,
> >>>>it is not really the cleanest option. The Debian guys did a port of
> >>>>NPTL using regular pthreads:
> >>>>
> >>
> >>Of course I ahould have meant "based on regular FreeBSD kernel services".
> >>
> >>>>http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
> >>>>
> >>>>I am certain this will require more research but it would be useful
> >>>>for other ports as well.
> >>>
> >>
> >>We could ask Petr Salinger for the details, but I am pretty sure
> >>FreeBSD has the required functionality natively.
> >>
> >>>Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
> >>>the lll stuff) in their pthread primitives which makes this
> >>>"possible". I call it gratuitous because it's really really hard to
> >>>achieve correct implementations of the pthread sync primitives that
> >>>don't have serious corner-case bugs, and it's unlikely that their
> >>>abstractions actually suffice to make correct alternate
> >>>implementations.
> >>>
> >>>musl does not have any such abstraction. We require a compare-and-swap
> >>>operation or equivalent on which arbitrary atomic operations can be
> >>>constructed, a futex or equivalent operation that's roughly
> >>>while(*addr==expected) sleep(), and implement all the sync primitives
> >>>just once on top of these.
> >>>
> >>
> >>I am not a threading expert (or even a CS guy), but it sounds like
> >>mutex(9) with condvar(9) would do [1]:
> >
> >No, they don't satisfy the needs of musl; they have their own
> >additional storage requirements and are probably not AS-safe. It might
> >be possible to use them to implement a userspace-emulated futex queue
> >(only if they are AS-safe), but I don't see a way to extend that to
> >the process-shared case.
> >
> 
> OK, it looks like sema(9) may be nearer (and also simpler but slower).
> For the process-shared case libthr(2) uses the stuff in sys/utmx.h,
> shich should be looked at but it is not documented[1].
> 
> Luckily Ed, the developer that would be mentoring the project, knows
> this stuff better than I do.

OK, well let's see if there are any proposals and what ideas people
come up with.

Rich

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

On 03/05/16 19:31, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 07:25:47PM -0500, Rich Felker wrote:
>> On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
>>>
>>>
>>> On 03/05/16 18:32, Rich Felker wrote:
>>>> On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
>>>>> First of all, great to hear there is interest on the musl side too.
>>>>>
>>>>> I think the biggest precedent of porting linux-oriented C libraries
>>>>> came from Debian's kFreeBSD. We accomodated a little by for them
>>>>> by defining __FreeBSD_kernel__ in sys/param.h.
>>>>>
>>>>> While using the optional linux-abi futex in FreeBSD could be an option,
>>>>> it is not really the cleanest option. The Debian guys did a port of
>>>>> NPTL using regular pthreads:
>>>>>
>>>
>>> Of course I ahould have meant "based on regular FreeBSD kernel services".
>>>
>>>>> http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
>>>>>
>>>>> I am certain this will require more research but it would be useful
>>>>> for other ports as well.
>>>>
>>>
>>> We could ask Petr Salinger for the details, but I am pretty sure
>>> FreeBSD has the required functionality natively.
>>>
>>>> Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
>>>> the lll stuff) in their pthread primitives which makes this
>>>> "possible". I call it gratuitous because it's really really hard to
>>>> achieve correct implementations of the pthread sync primitives that
>>>> don't have serious corner-case bugs, and it's unlikely that their
>>>> abstractions actually suffice to make correct alternate
>>>> implementations.
>>>>
>>>> musl does not have any such abstraction. We require a compare-and-swap
>>>> operation or equivalent on which arbitrary atomic operations can be
>>>> constructed, a futex or equivalent operation that's roughly
>>>> while(*addr==expected) sleep(), and implement all the sync primitives
>>>> just once on top of these.
>>>>
>>>
>>> I am not a threading expert (or even a CS guy), but it sounds like
>>> mutex(9) with condvar(9) would do [1]:
>>
>> No, they don't satisfy the needs of musl; they have their own
>> additional storage requirements and are probably not AS-safe. It might
>> be possible to use them to implement a userspace-emulated futex queue
>> (only if they are AS-safe), but I don't see a way to extend that to
>> the process-shared case.
>
> Actually these look like kernelspace APIs, not userspace, unless I'm
> mistaken. In that case I don't see how they're relevant/usable. Is tht
> correct?
>

Ugh .. yeah ... sem(9) is kernel only API too.

The best is to look at the native thread library, not precisely
the easiest to read code in the world, I'm afraid.

OpenGrok may help:

http://src.illumos.org/source/xref/freebsd-head/lib/libthr/

Pedro.

Re: FreeBSD's Google Summer of Code 2016

[Felix Fietkau]

On 2016-03-06 01:25, Rich Felker wrote:
> On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
>> 
>> 
>> On 03/05/16 18:32, Rich Felker wrote:
>> >On Sat, Mar 05, 2016 at 05:41:25PM -0500, Pedro Giffuni wrote:
>> >>First of all, great to hear there is interest on the musl side too.
>> >>
>> >>I think the biggest precedent of porting linux-oriented C libraries
>> >>came from Debian's kFreeBSD. We accomodated a little by for them
>> >>by defining __FreeBSD_kernel__ in sys/param.h.
>> >>
>> >>While using the optional linux-abi futex in FreeBSD could be an option,
>> >>it is not really the cleanest option. The Debian guys did a port of
>> >>NPTL using regular pthreads:
>> >>
>> 
>> Of course I ahould have meant "based on regular FreeBSD kernel services".
>> 
>> >>http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
>> >>
>> >>I am certain this will require more research but it would be useful
>> >>for other ports as well.
>> >
>> 
>> We could ask Petr Salinger for the details, but I am pretty sure
>> FreeBSD has the required functionality natively.
>> 
>> >Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
>> >the lll stuff) in their pthread primitives which makes this
>> >"possible". I call it gratuitous because it's really really hard to
>> >achieve correct implementations of the pthread sync primitives that
>> >don't have serious corner-case bugs, and it's unlikely that their
>> >abstractions actually suffice to make correct alternate
>> >implementations.
>> >
>> >musl does not have any such abstraction. We require a compare-and-swap
>> >operation or equivalent on which arbitrary atomic operations can be
>> >constructed, a futex or equivalent operation that's roughly
>> >while(*addr==expected) sleep(), and implement all the sync primitives
>> >just once on top of these.
>> >
>> 
>> I am not a threading expert (or even a CS guy), but it sounds like
>> mutex(9) with condvar(9) would do [1]:
> 
> No, they don't satisfy the needs of musl; they have their own
> additional storage requirements and are probably not AS-safe. It might
> be possible to use them to implement a userspace-emulated futex queue
> (only if they are AS-safe), but I don't see a way to extend that to
> the process-shared case.
What about umtx? It's supposed to be just like linux futex.

- Felix

Re: FreeBSD's Google Summer of Code 2016

[Szabolcs Nagy]

* Felix Fietkau <nbd@openwrt.org> [2016-03-07 09:32:17 +0100]:
> What about umtx? It's supposed to be just like linux futex.
> 

i see
https://svnweb.FreeBSD.org/base/head/sys/sys/umtx.h?view=markup

but there seems to be no documentation for it in
https://www.freebsd.org/cgi/man.cgi

if _umtx_op does what it seems to do, then i think musl can be
ported to native freebsd syscalls, but i don't know how much
syscall abi stability freebsd is willing to guarantee.
i think historically that was assumed to be a private interface
between the libc and the kernel on bsd (in which case a musl port
cannot be easily maintained outside of the bsd source repo).

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

On 03/07/16 03:32, Felix Fietkau wrote:
> On 2016-03-06 01:25, Rich Felker wrote:
>> On Sat, Mar 05, 2016 at 07:14:34PM -0500, Pedro Giffuni wrote:
... <snip>

>>> Of course I ahould have meant "based on regular FreeBSD kernel services".
>>>
>>>>> http://thread.gmane.org/gmane.linux.debian.ports.bsd/11702
>>>>>
>>>>> I am certain this will require more research but it would be useful
>>>>> for other ports as well.
>>>>
>>>
>>> We could ask Petr Salinger for the details, but I am pretty sure
>>> FreeBSD has the required functionality natively.
>>>
>>>> Glibc/NPTL has a lot of what I'd call "gratuitous abstraction" (like
>>>> the lll stuff) in their pthread primitives which makes this
>>>> "possible". I call it gratuitous because it's really really hard to
>>>> achieve correct implementations of the pthread sync primitives that
>>>> don't have serious corner-case bugs, and it's unlikely that their
>>>> abstractions actually suffice to make correct alternate
>>>> implementations.
>>>>
>>>> musl does not have any such abstraction. We require a compare-and-swap
>>>> operation or equivalent on which arbitrary atomic operations can be
>>>> constructed, a futex or equivalent operation that's roughly
>>>> while(*addr==expected) sleep(), and implement all the sync primitives
>>>> just once on top of these.
>>>>
>>>
>>> I am not a threading expert (or even a CS guy), but it sounds like
>>> mutex(9) with condvar(9) would do [1]:
>>
>> No, they don't satisfy the needs of musl; they have their own
>> additional storage requirements and are probably not AS-safe. It might
>> be possible to use them to implement a userspace-emulated futex queue
>> (only if they are AS-safe), but I don't see a way to extend that to
>> the process-shared case.
> What about umtx? It's supposed to be just like linux futex.
>

Yes, that's it.

It is not well documented beyond the <sys/umtx.h>  header but it
is indeed used to implement the linux abi futex and the native
libthr on FreeBSD.

Pedro.

Re: FreeBSD's Google Summer of Code 2016

[Pedro Giffuni]

Hello;

On 03/07/16 08:06, Szabolcs Nagy wrote:
> * Felix Fietkau <nbd@openwrt.org> [2016-03-07 09:32:17 +0100]:
>> What about umtx? It's supposed to be just like linux futex.
>>
>
> i see
> https://svnweb.FreeBSD.org/base/head/sys/sys/umtx.h?view=markup
>
> but there seems to be no documentation for it in
> https://www.freebsd.org/cgi/man.cgi
>
> if _umtx_op does what it seems to do, then i think musl can be
> ported to native freebsd syscalls, but i don't know how much
> syscall abi stability freebsd is willing to guarantee.
> i think historically that was assumed to be a private interface
> between the libc and the kernel on bsd (in which case a musl port
> cannot be easily maintained outside of the bsd source repo).
>

I wouldn't worry about ABI stability. We are not allowed to break
the ABI per branch and with so many users depending on being able
to run previous versions of FreeBSD within jails, and with the
native threads and now linux binaries depending on it, there is
no chance we can break it,

We may introduce a new interface in the future (who knows) but we
can't just drop an old one.

Pedro.