Custom __set_thread_area for ARM

Custom __set_thread_area for ARM

[Oleg Ranevskyy]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

Dear community,

Musl has a generic implementation of the __set_thread_area function in
src/thread/__set_thread_area.c. It is not used for ARM though. There is a
custom ARM implementation in src/thread/arm/__set_thread_area.s.

Would you be able to clarify the following question please?
Why musl doesn't define SYS_set_thread_area for ARM to utilize the generic
function and uses custom __set_thread_area instead?

Thank you in advance for any help.
Oleg

[-- Attachment #2: Type: text/html, Size: 575 bytes --]

Re: Custom __set_thread_area for ARM

[Rich Felker]

On Tue, Jan 13, 2015 at 11:19:44PM +0300, Oleg Ranevskyy wrote:
> Dear community,
> 
> Musl has a generic implementation of the __set_thread_area function in
> src/thread/__set_thread_area.c. It is not used for ARM though. There is a
> custom ARM implementation in src/thread/arm/__set_thread_area.s.
> 
> Would you be able to clarify the following question please?
> Why musl doesn't define SYS_set_thread_area for ARM to utilize the generic
> function and uses custom __set_thread_area instead?

The ARM kernel does not implement SYS_set_thread_area. Instead it
provides an ARM-specific syscall. The asm file you're looking at uses
that instead.

BTW, this code is replaced in git master and the pending 1.1.6
release. It's part of the ARM atomics/TLS access overhaul.

Rich

Re: Custom __set_thread_area for ARM

[Andy Lutomirski]

On 01/13/2015 07:09 PM, Rich Felker wrote:
> On Tue, Jan 13, 2015 at 11:19:44PM +0300, Oleg Ranevskyy wrote:
>> Dear community,
>>
>> Musl has a generic implementation of the __set_thread_area function in
>> src/thread/__set_thread_area.c. It is not used for ARM though. There is a
>> custom ARM implementation in src/thread/arm/__set_thread_area.s.
>>
>> Would you be able to clarify the following question please?
>> Why musl doesn't define SYS_set_thread_area for ARM to utilize the generic
>> function and uses custom __set_thread_area instead?
> 
> The ARM kernel does not implement SYS_set_thread_area. Instead it
> provides an ARM-specific syscall. The asm file you're looking at uses
> that instead.
> 
> BTW, this code is replaced in git master and the pending 1.1.6
> release. It's part of the ARM atomics/TLS access overhaul.

As the sort-of-maintainer of the kernel side of this on x86, I have to
ask: why is the i386 __set_thread_area function written in assembly?

--Andy

Re: Re: Custom __set_thread_area for ARM

[Rich Felker]

On Wed, Jan 21, 2015 at 05:45:56PM -0800, Andy Lutomirski wrote:
> On 01/13/2015 07:09 PM, Rich Felker wrote:
> > On Tue, Jan 13, 2015 at 11:19:44PM +0300, Oleg Ranevskyy wrote:
> >> Dear community,
> >>
> >> Musl has a generic implementation of the __set_thread_area function in
> >> src/thread/__set_thread_area.c. It is not used for ARM though. There is a
> >> custom ARM implementation in src/thread/arm/__set_thread_area.s.
> >>
> >> Would you be able to clarify the following question please?
> >> Why musl doesn't define SYS_set_thread_area for ARM to utilize the generic
> >> function and uses custom __set_thread_area instead?
> > 
> > The ARM kernel does not implement SYS_set_thread_area. Instead it
> > provides an ARM-specific syscall. The asm file you're looking at uses
> > that instead.
> > 
> > BTW, this code is replaced in git master and the pending 1.1.6
> > release. It's part of the ARM atomics/TLS access overhaul.
> 
> As the sort-of-maintainer of the kernel side of this on x86, I have to
> ask: why is the i386 __set_thread_area function written in assembly?

First, there's the musl-technical answer: the build system's per-arch
replacement files are asm, not C. It's possible to use C by making an
empty asm file and putting the replacement C file in the arch dir, but
it's not as obvious what's going on. This could be changed if it
helped, but...

For __set_thread_area, there are good reasons for it to be asm. The
x86 set_thread_area syscall is not usable without asm because you have
to load the resulting segment into %gs. And as for musl in particular,
we don't want an arch-specific function signature like the kernel has
for this one on x86, taking a pointer to a user_desc struct. We want
the function to simply take the desired thread pointer value and load
it. On some archs that doesn't even need a syscall; it's just loading
the argument into a GPR. On x86, however, it requires setting up a
user_desc structure, passing that to the kernel, then loading %gs
based on the result. Since we also want binaries that don't crash on
ancient (2.4) kernels (even though they can't support threads), we
also need the fallback code to use the modify_ldt syscall when
set_thread_area is not available.

BTW you can find some documentation of the history of musl's
__set_thread_area via:

git log -p src/thread/i386/__set_thread_area.s

Rich

Re: Custom __set_thread_area for ARM

[Andy Lutomirski]

On 01/21/2015 09:10 PM, Rich Felker wrote:
> On Wed, Jan 21, 2015 at 05:45:56PM -0800, Andy Lutomirski wrote:
>> On 01/13/2015 07:09 PM, Rich Felker wrote:
>>> On Tue, Jan 13, 2015 at 11:19:44PM +0300, Oleg Ranevskyy wrote:
>>>> Dear community,
>>>>
>>>> Musl has a generic implementation of the __set_thread_area function in
>>>> src/thread/__set_thread_area.c. It is not used for ARM though. There is a
>>>> custom ARM implementation in src/thread/arm/__set_thread_area.s.
>>>>
>>>> Would you be able to clarify the following question please?
>>>> Why musl doesn't define SYS_set_thread_area for ARM to utilize the generic
>>>> function and uses custom __set_thread_area instead?
>>>
>>> The ARM kernel does not implement SYS_set_thread_area. Instead it
>>> provides an ARM-specific syscall. The asm file you're looking at uses
>>> that instead.
>>>
>>> BTW, this code is replaced in git master and the pending 1.1.6
>>> release. It's part of the ARM atomics/TLS access overhaul.
>>
>> As the sort-of-maintainer of the kernel side of this on x86, I have to
>> ask: why is the i386 __set_thread_area function written in assembly?
> 
> First, there's the musl-technical answer: the build system's per-arch
> replacement files are asm, not C. It's possible to use C by making an
> empty asm file and putting the replacement C file in the arch dir, but
> it's not as obvious what's going on. This could be changed if it
> helped, but...
> 
> For __set_thread_area, there are good reasons for it to be asm. The
> x86 set_thread_area syscall is not usable without asm because you have
> to load the resulting segment into %gs. And as for musl in particular,
> we don't want an arch-specific function signature like the kernel has
> for this one on x86, taking a pointer to a user_desc struct. We want
> the function to simply take the desired thread pointer value and load
> it. On some archs that doesn't even need a syscall; it's just loading
> the argument into a GPR. On x86, however, it requires setting up a
> user_desc structure, passing that to the kernel, then loading %gs
> based on the result. Since we also want binaries that don't crash on
> ancient (2.4) kernels (even though they can't support threads), we
> also need the fallback code to use the modify_ldt syscall when
> set_thread_area is not available.
> 
> BTW you can find some documentation of the history of musl's
> __set_thread_area via:
> 
> git log -p src/thread/i386/__set_thread_area.s

Does musl not use inline asm?  ISTM something like:

struct user_desc desc;
memset(&desc, 0, sizeof(desc));
desc.base = whatever;
// assign other fields
if (set_thread_area(&desc) != 0)
    handle error;

asm volatile ("mov %0,%%fs" : : "=rm" ((desc.entry_number << 3) | 3));

would be a lot more comprehensible.

--Andy

Re: Re: Custom __set_thread_area for ARM

[Szabolcs Nagy]

* Andy Lutomirski <luto@amacapital.net> [2015-01-27 15:32:27 -0800]:
> Does musl not use inline asm?  ISTM something like:
> 

it is arch specific code, which is currently either

- in asm in src/DIR/ARCH/foo.s if there is appropriate src/DIR/foo.c
- in c in a header under arch/ARCH
- in c in arch/ARCH/src/foo.c
- in asm in arch/ARCH/src/ARCH/foo.s if there is appropriate foo.c

so under src/thread you cannot have arch specific c code
with the current makefile, it has to be asm there

> struct user_desc desc;
> memset(&desc, 0, sizeof(desc));
> desc.base = whatever;
> // assign other fields
> if (set_thread_area(&desc) != 0)
>     handle error;
> 
> asm volatile ("mov %0,%%fs" : : "=rm" ((desc.entry_number << 3) | 3));
> 
> would be a lot more comprehensible.
> 
> --Andy

Re: Re: Custom __set_thread_area for ARM

[Rich Felker]

On Tue, Jan 27, 2015 at 03:32:27PM -0800, Andy Lutomirski wrote:
> Does musl not use inline asm?  ISTM something like:

At present, only in places where it's actually intended to be inlined
efficiently -- that mainly means syscalls and reading the thread
pointer. Both of these are places where it's just an optimization, so
it could be turned off and replaced by an external function if needed.
There has been some interest in making it easy to do that to work with
compilers that don't have any GNU inline asm support, though it's not
at this time a topic of much practical interest to me since most/all
relevant compilers seem to support it.

Also, it's worth noting that some of the other things that actually
need asm (rather than just using it as an optimization) also need it
to external: especially setjmp and vfork cannot be implemented in
inline asm no matter what you do.

> struct user_desc desc;
> memset(&desc, 0, sizeof(desc));
> desc.base = whatever;
> // assign other fields
> if (set_thread_area(&desc) != 0)
>     handle error;
> 
> asm volatile ("mov %0,%%fs" : : "=rm" ((desc.entry_number << 3) | 3));
> 
> would be a lot more comprehensible.

I can see both viewpoints here, but I think the functions where a mix
of C/asm is "more comprehensible" than just asm are a fairly small
set, the difference in readability is probably small, and the compiler
would probably generate moderately larger code (large
prologue/epilogue/stack frames, etc.) so in the absence of a strong
motivation to change it I think this is probably best left alone. Note
on that x86 is pretty unique in having anything that could be made
more readable by mixing C. The other archs have fairly straightforward
set_thread_area -- either loading a GPR or making an arch-specific
syscall.

Rich