Re: [musl] Draft riscv64 TLSDESC implementation

[Rich Felker <dalias@libc.org>]

On Tue, Jan 23, 2024 at 12:52:01AM -0800, Fangrui Song wrote:
> On Sun, Jan 21, 2024 at 7:41 PM Tatsuyuki Ishi <ishitatsuyuki@gmail..com> wrote:
> >
> > > On Jan 22, 2024, at 9:03, Rich Felker <dalias@libc.org> wrote:
> > >
> > > On Sun, Jan 21, 2024 at 03:48:55PM -0800, Fangrui Song wrote:
> > >> On Sun, Jan 21, 2024 at 2:28 PM Rich Felker <dalias@libc.org> wrote:
> > >>>
> > >>> On Tue, Aug 22, 2023 at 01:38:21PM -0400, Rich Felker wrote:
> > >>>> The psABI work is not finalized, but based on the current status of
> > >>>> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/373, I think
> > >>>> the attached is a valid (but untested) implementation of TLSDESC for
> > >>>> riscv64. Actually activating it requires also adding the relocation
> > >>>> type macro to riscv64/reloc.h.
> > >>>>
> > >>>> If any rv folks could look it over and make sure I haven't made any
> > >>>> stupid asm errors or missed any obvious optimizations, that would help
> > >>>> to quickly get this merged when the psABI is finalized.
> > >>>>
> > >>>> Rich
> > >>>
> > >>>> .text
> > >>>> .global __tlsdesc_static
> > >>>> .hidden __tlsdesc_static
> > >>>> .type __tlsdesc_static,%function
> > >>>> __tlsdesc_static:
> > >>>>      ld a0,8(a0)
> > >>>>      jr t0
> > >>>>
> > >>>> .global __tlsdesc_dynamic
> > >>>> .hidden __tlsdesc_dynamic
> > >>>> .type __tlsdesc_dynamic,%function
> > >>>> __tlsdesc_dynamic:
> > >>>>      add sp,sp,-8
> > >>>>      sd t1,(sp)
> > >>>>      sd t2,8(sp)
> > >>>>
> > >>>>      ld t2,-8(tp) # t2=dtv
> > >>>>
> > >>>>      ld a0,8(a0)  # a0=&{modidx,off}
> > >>>>      ld t1,8(a0)  # t1=off
> > >>>>      ld a0,(a0)   # a0=modidx
> > >>>>      sll a0,a0,3  # a0=8*modidx
> > >>>>
> > >>>>      add a0,a0,t2 # a0=dtv+8*modidx
> > >>>>      ld a0,(a0)   # a0=dtv[modidx]
> > >>>>      add a0,a0,t1 # a0=dtv[modidx]+off
> > >>>>      sub a0,a0,tp # a0=dtv[modidx]+off-tp
> > >>>>
> > >>>>      ld t1,(sp)
> > >>>>      ld t2,8(sp)
> > >>>>      add sp,sp,8
> > >>>>      jr t0
> > >>>
> > >>> Any feedback on this? Offhand, it looks like adjusting sp by 8 is
> > >>> wrong and that should be 16. Anything else? Does anyone have recent
> > >>> enough tooling to test this?
> > >>
> > >> Tatsuyuki, do you have links to the latest version of
> > >> gcc/binutils/glibc patches?
> > >> Downloading patches from these mailing lists is probably a large
> > >> hurdle for many users, so having the relevant repositories online may
> > >> help.
> > >>
> > >> mold has implemented RISC-V TLSDESC.
> > >>
> > >> On the LLVM side, I have reviewed
> > >> https://github.com/llvm/llvm-project/pull/66915 and am waiting for it
> > >> to land, before I can check the lld status.
> > >
> > > To test this, drop it in src/ldso/riscv64/tlsdesc.s, and add to
> > > arch/riscv64/reloc.h:
> > >
> > > #define REL_TLSDESC R_RISCV_TLSDESC
> > >
> > > or whatever the reloc name is (I don't think it's in elf.h yet so you
> > > probably need to either add it there too or just hard-code the number
> > > for testing).
> > >
> > > Updated version with the sp bugfix attached.
> >
> > The assembly looks fine to me. (It’s nice that musl don’t need to bother with save/restore at all since DTVs are initialized eagerly..)
> > For the patches mentioned in the other thread, most tests were done with glibc’s portable testsuite (tst-elf-*).
> > If musl has a similar one, you should be able to run it with my GCC / binutils fork (with either --with_tls=desc at configure time or -mtls-dialect=desc at compile time).
> >
> > Tatsuyuki.
> >
> > > Rich
> > > <tlsdesc.s>
> >
> 
> I have verified that the patch works using a runtime test under qemu-user.
> I use Paul Kirth's pending LLVM codegen/assembly patch and my pending
> lld patch:)

Thanks for running tests!

> Compile b.c to bb.s. Replace general dynamic code sequences (e.g.
> `la.tls.gd a0,tls0; call __tls_get_addr@plt`) with TLSDESC, e.g.
> ```
> ..Ltlsdesc_hi0:
>   auipc a0, %tlsdesc_hi(tls0)
>   ld  a1, %tlsdesc_load_lo(.Ltlsdesc_hi0)(a0)
>   addi  a0, a0, %tlsdesc_add_lo(.Ltlsdesc_hi0)
>   jalr  t0, 0(a1), %tlsdesc_call(.Ltlsdesc_hi0)
>   add   a0, a0, tp
> ```

Is this due to compiler not supporting generation of TLSDESC sequence
yet? If so, unlike a full test with compiler support, it does not test
for the TLSDESC machinery honoring all the non-clobbering requirements
of the ABI. But I think we can check by hand that that part of the ABI
is honored.

Rich