From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id 66D8E250F9 for ; Tue, 23 Jan 2024 09:52:35 +0100 (CET) Received: (qmail 30474 invoked by uid 550); 23 Jan 2024 08:50:28 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 30439 invoked from network); 23 Jan 2024 08:50:27 -0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ee93YfqCnSxkxQR3ikvdd3z74APyj6Hdg1PHwxet11n/wixqNMWgqGP95xiIG3nGYQURsTOv9ghneL7AzpD+q16CyYn56RC8hsDx7DZgizvbbY7AaJ5LJ+MpokqDiTgzVCnKNhtjAGnS1IUDUv08zlgainojDkF3C8UBr1bpERbkhKj6zB9qsuh/hHX5mAvjRFLI3rr/Zk8kzeMMeSrm2K2ifYkqFK3qFS8iKdmOvPqCh1DdN/yMTW6JpYyDZRCUOoaMzFZ2lb4ziUvPIYM2U2X8bUtcbeSZOB+kPqcf9IkpXkJ78zpjLrqToBfb9KrkQBSrbjgusGgl/H9DnIcaaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z4Ik2AqopsE0c/ZC5UelEMu5HrEK2oY5cisDMzpT/RY=; b=TICqXAI6O64oxWAYaA/wuXPwmZaqD1/hwo0O+zOL60BsVq93nhbPvHJVgOl50Vx4LNASIantL9kCSfoufzWwpMAqNmQEf7vrNBgwRCjnR8z3/ZvY7BXIs8Oz99ZvhP+7UVvCzsilOz/pVKgGe5I3UWP9uuE2YaQmKuleZjN7Buoa2W8v1nfDVwgO6w8VWLDtjv1B7ugsC7dcI/FTAikL+sN+c68nooPOZzLnC/UttXVYOVfWtm3VY3+mLCTZg5erlYZNYyeAwmgzDNRnzJ0zB9ZUZPyoASoW6RoCSAkeXK8l3Pv3xLEMoLJLgx/ODPk0qiDIQJlWlz+eEO6UIb1SRA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none X-Gm-Message-State: AOJu0YxR1mwN73po61GyOllh9cJci5wU+wnK85U9TBj54aj75wtieAoe piByA33I3zbLtsIY5e+Qn2xxP3fHfUafd1tCw1QiKDROUeY+OID+SH123yVynxHVrMlmn64TLNI VDyLp1O0cey3MvNmqrBL+5qGRNJs= X-Google-Smtp-Source: AGHT+IHkV7LLZIpe3+t3cv3vXffrgM0KfeJlQF50vjA49ZFAhW+gXy6nD8oP/ypq0g+0AHbh/K8ivlcmP56mc5+A6sU= X-Received: by 2002:a05:6122:da3:b0:4b6:e60e:e080 with SMTP id bc35-20020a0561220da300b004b6e60ee080mr1841365vkb.30.1705999932449; Tue, 23 Jan 2024 00:52:12 -0800 (PST) References: <20230822173821.GW4163@brightrain.aerifal.cx> <20240121222831.GC4163@brightrain.aerifal.cx> <20240122000348.GD4163@brightrain.aerifal.cx> In-Reply-To: From: Fangrui Song Date: Tue, 23 Jan 2024 00:52:01 -0800 X-Gmail-Original-Message-ID: Message-ID: To: Rich Felker Cc: musl@lists.openwall.com, Tatsuyuki Ishi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-TMN: [qltA8s3ilc2BP9cdNPEnTludAc6AN9TB] X-ClientProxiedBy: JH0PR01CA0096.apcprd01.prod.exchangelabs.com (2603:1096:990:58::10) To DS7PR12MB5765.namprd12.prod.outlook.com (2603:10b6:8:74::19) X-Microsoft-Original-Message-ID: MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB5765:EE_|IA1PR12MB7494:EE_ X-MS-Office365-Filtering-Correlation-Id: 8771c7f6-a5bc-43ba-8cdc-08dc1bf09b48 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6DTML5imQUqOCpJSqeN5bdE7O5C+yp+dBugLEzbAinLLXB2zbdpBlKtUzUXYHDy9tIYly6aRISVLo+mp056GsyuYGJNvyEtTrIz9I/CS5CECvVyggmlLksVjueFyAjWOM5R8K5jbbIw0CUsJugN/rIDZXjU6gVj6sJgmm5NL3qi3fMU61zsOiM77v9QhOkFg9y4nSMQQwGFjUSlCtHIXmBHL7KB85dcORQDF7dvmk43Lma9nDCpBzyXabd19aqtQnPOoRinE94arObHUGcF7ikb+dDaq1kAUz+zJZRLeQK3DVCO4XWrj4UwH88P9gKu/qgEG4CJiwR9/gHAV50Q0sqoN/0wzi/PwPUy3F520f8BU6fReD9CTQbGpT7cmun2h0MGEZwMdSeJ2gnpb/RU8zgQpDFw6n7TXIFKbqiNOEjLM/GmiUkQwDkr2IfO1IhDEYOaR7TmU6x3EGyqHtgBtKVpXq7Y5Z2JVUcY7crmNjOTxIVl2YR+CKgSuosNwvAVVQy71ha3WFgWyF9KxBu9xR3vPWWvcoFug6OyCdfHHqMel33rx5LQxIYIYF92YIFh5kiaI+h7c42zFOaMyOX9ukM2s+9hxhQ7uueBt6/A0VUUcsdqozNzSN3UicgAU+5ngzXD3xno42vuWFnT+XnrmOQ== X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?clhCaUNiV090a1pkTnNTeU03eTBIdERhSXFqY2pISGlySmg3djBUb0QxQnJF?= =?utf-8?B?NHpLRjZYejNJbkt6UGlsOXlNZmtpbVAycmoxTDFhbnBtbk15aDJpZ1g2S3hp?= =?utf-8?B?SHVxc0taaWd0UmJVUCtDV0xVdTFJcnlpSmU3TmpiOS9IREVZMVRkYTd1c0Zi?= =?utf-8?B?TTdEL3FVbW1WQkdiODdRZkxZM1dSOWZzSEpocDhTTDZYanlvZnc0Yk1HekVl?= =?utf-8?B?RVQ4TGlNdUtQV2dVMGZSOURjazJhVHFDM2Z0dkMrS1oyazhtSE82bG9oeWl2?= =?utf-8?B?Q21idCtIZC9TNEtGWGRSWEdSTUoyNXNMTmlnbFBGM21MMWJzaDNzcVE5WGx5?= =?utf-8?B?YThtUUtMMlRWM0dSRzlES0VMRnlaQThUZVBDdVZ4UmY5aUNsQ0RXd2V3UmxK?= =?utf-8?B?NEg0bzVrNHU3K1BQcWlxNkoxU2tlcjlqWS9EeHhKSnQ3MTRsRGZTdVphWWE0?= =?utf-8?B?c3BJSWd6SmxuNXE2OVVzcS9sOTdWOUEvazFPdEc5NC92aGxZdGgzbVQwWkts?= =?utf-8?B?VGE1VmRraDRPcnV6d0g1QzFkalArcEM0WERSa1lSRWhzdlRXL3cwZUVmWHdo?= =?utf-8?B?OVlGbm9GdHRTRUIvWlhjYUUrUCt0VTVzeFEzbnBWWTgwNHhrNzRYOGp1R2Vi?= =?utf-8?B?YVM1VlhTcFVUVE82ZzhzNVN1UXNpVWJTNkZoLy9ndVlrMjlMR2llQmpJWUY4?= =?utf-8?B?YUcvZDkvS3dmTnZkbjRoY0VXUUJSYzNiOFVoNEFCS0dFYkhRRm9LM0RzcmZs?= =?utf-8?B?SXg0L3pySS83NjdRMm13N1pHYThyTTJHY3JmRWE0dDdRNWhDbHFvZytFUm96?= =?utf-8?B?Z0oraVNiTzNxTndMZU9FWThqajBEdTNTZlByRGMxdFZJM3BmV3pKelhSMjBS?= =?utf-8?B?Q1FubVdJeGtVeUpibCt6K3NMb0I2WHprUE1WR0hSWXZ6eXlCRnYxa1liU2Jy?= =?utf-8?B?d3hua0F0Tkg3SVdTRm1vT0pMTUYxbUFtT0VlZk1iWGk3Y2JrYXBDdERoSDA4?= =?utf-8?B?SWtrckdQajlGd0NBTnhpZ1RZdVJLbnE4VTM2V3dMMG4wMGo3SGNkZTYza1hj?= =?utf-8?B?UHFrQUVWbmtUbHEzOEl4UGlHYU5QaW1WandLb1M2REVPbTdVREhlWVJqZEVa?= =?utf-8?B?RER3dUFDeHVNQTJ0cE1IbHVPTFQrQkhhbWNINzQ2eTVRQ1JRQzEwc1RrTzBw?= =?utf-8?B?amhLN3pSVFpHV3JHajBuN0dQbTlYenJUL3lFd0ZhVzVMREdGTzJ4bkFJRDZM?= =?utf-8?B?Y0hnSkFTdWV4QXVkb2ZhcTNxczRmU0JWdlhRRy9uc2laWnNXWjM0TkJFR1FE?= =?utf-8?B?Z1kzZUlUWjd2ZUQrcnJFOS95NG82L0ZHU1daZE9ZcWdFQUNUKzlRTGRpYjBp?= =?utf-8?B?N1JDK2NjbGRPVkJFaGJrRmkyMjR3VGgyOE00dzNCLzZXbE5nQWFaSVdKS3B0?= =?utf-8?B?Y0wrQU9uTGg0QnkycjdsMmtOblhselJoTGdDL2ZtOFk3UkFFbGFCSjRqT2c1?= =?utf-8?B?MkNIT0NPN0JvaE9ObzhIR3c4SjIyYVovTDRHQXRwMWYyOGNJREowTU5qaERy?= =?utf-8?Q?TnXKs0p07k7zWlPFJMPp+x4/5+jWaILlpeueh5HzH9++77?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-71ea3.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 8771c7f6-a5bc-43ba-8cdc-08dc1bf09b48 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB5765.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2024 08:52:19.3042 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7494 Subject: Re: [musl] Draft riscv64 TLSDESC implementation On Sun, Jan 21, 2024 at 7:41=E2=80=AFPM Tatsuyuki Ishi wrote: > > > On Jan 22, 2024, at 9:03, Rich Felker wrote: > > > > On Sun, Jan 21, 2024 at 03:48:55PM -0800, Fangrui Song wrote: > >> On Sun, Jan 21, 2024 at 2:28=E2=80=AFPM Rich Felker = wrote: > >>> > >>> On Tue, Aug 22, 2023 at 01:38:21PM -0400, Rich Felker wrote: > >>>> The psABI work is not finalized, but based on the current status of > >>>> https://github.com/riscv-non-isa/riscv-elf-psabi-doc/pull/373, I thi= nk > >>>> the attached is a valid (but untested) implementation of TLSDESC for > >>>> riscv64. Actually activating it requires also adding the relocation > >>>> type macro to riscv64/reloc.h. > >>>> > >>>> If any rv folks could look it over and make sure I haven't made any > >>>> stupid asm errors or missed any obvious optimizations, that would he= lp > >>>> to quickly get this merged when the psABI is finalized. > >>>> > >>>> Rich > >>> > >>>> .text > >>>> .global __tlsdesc_static > >>>> .hidden __tlsdesc_static > >>>> .type __tlsdesc_static,%function > >>>> __tlsdesc_static: > >>>> ld a0,8(a0) > >>>> jr t0 > >>>> > >>>> .global __tlsdesc_dynamic > >>>> .hidden __tlsdesc_dynamic > >>>> .type __tlsdesc_dynamic,%function > >>>> __tlsdesc_dynamic: > >>>> add sp,sp,-8 > >>>> sd t1,(sp) > >>>> sd t2,8(sp) > >>>> > >>>> ld t2,-8(tp) # t2=3Ddtv > >>>> > >>>> ld a0,8(a0) # a0=3D&{modidx,off} > >>>> ld t1,8(a0) # t1=3Doff > >>>> ld a0,(a0) # a0=3Dmodidx > >>>> sll a0,a0,3 # a0=3D8*modidx > >>>> > >>>> add a0,a0,t2 # a0=3Ddtv+8*modidx > >>>> ld a0,(a0) # a0=3Ddtv[modidx] > >>>> add a0,a0,t1 # a0=3Ddtv[modidx]+off > >>>> sub a0,a0,tp # a0=3Ddtv[modidx]+off-tp > >>>> > >>>> ld t1,(sp) > >>>> ld t2,8(sp) > >>>> add sp,sp,8 > >>>> jr t0 > >>> > >>> Any feedback on this? Offhand, it looks like adjusting sp by 8 is > >>> wrong and that should be 16. Anything else? Does anyone have recent > >>> enough tooling to test this? > >> > >> Tatsuyuki, do you have links to the latest version of > >> gcc/binutils/glibc patches? > >> Downloading patches from these mailing lists is probably a large > >> hurdle for many users, so having the relevant repositories online may > >> help. > >> > >> mold has implemented RISC-V TLSDESC. > >> > >> On the LLVM side, I have reviewed > >> https://github.com/llvm/llvm-project/pull/66915 and am waiting for it > >> to land, before I can check the lld status. > > > > To test this, drop it in src/ldso/riscv64/tlsdesc.s, and add to > > arch/riscv64/reloc.h: > > > > #define REL_TLSDESC R_RISCV_TLSDESC > > > > or whatever the reloc name is (I don't think it's in elf.h yet so you > > probably need to either add it there too or just hard-code the number > > for testing). > > > > Updated version with the sp bugfix attached. > > The assembly looks fine to me. (It=E2=80=99s nice that musl don=E2=80=99t= need to bother with save/restore at all since DTVs are initialized eagerly= .) > For the patches mentioned in the other thread, most tests were done with = glibc=E2=80=99s portable testsuite (tst-elf-*). > If musl has a similar one, you should be able to run it with my GCC / bin= utils fork (with either --with_tls=3Ddesc at configure time or -mtls-dialec= t=3Ddesc at compile time). > > Tatsuyuki. > > > Rich > > > I have verified that the patch works using a runtime test under qemu-user. I use Paul Kirth's pending LLVM codegen/assembly patch and my pending lld patch:) https://gist.github.com/MaskRay/7ad19393fdb0834540db6e7b6b02fa56 . Copied instructions below: Patch musl ```diff --- c/arch/riscv64/reloc.h +++ i/arch/riscv64/reloc.h @@ -17,6 +17,7 @@ #define REL_DTPMOD R_RISCV_TLS_DTPMOD64 #define REL_DTPOFF R_RISCV_TLS_DTPREL64 #define REL_TPOFF R_RISCV_TLS_TPREL64 +#define REL_TLSDESC 12 #define CRTJMP(pc,sp) __asm__ __volatile__( \ "mv sp, %1 ; jr %0" : : "r"(pc), "r"(sp) : "memory" ) diff --git c/src/ldso/riscv64/tlsdesc.s i/src/ldso/riscv64/tlsdesc.s new file mode 100644 index 00000000..56d1ce89 --- /dev/null +++ i/src/ldso/riscv64/tlsdesc.s @@ -0,0 +1,33 @@ +.text +.global __tlsdesc_static +.hidden __tlsdesc_static +.type __tlsdesc_static,%function +__tlsdesc_static: + ld a0,8(a0) + jr t0 + +.global __tlsdesc_dynamic +.hidden __tlsdesc_dynamic +.type __tlsdesc_dynamic,%function +__tlsdesc_dynamic: + add sp,sp,-16 + sd t1,(sp) + sd t2,8(sp) + + ld t2,-8(tp) # t2=3Ddtv + + ld a0,8(a0) # a0=3D&{modidx,off} + ld t1,8(a0) # t1=3Doff + ld a0,(a0) # a0=3Dmodidx + sll a0,a0,3 # a0=3D8*modidx + + add a0,a0,t2 # a0=3Ddtv+8*modidx + ld a0,(a0) # a0=3Ddtv[modidx] + add a0,a0,t1 # a0=3Ddtv[modidx]+off + sub a0,a0,tp # a0=3Ddtv[modidx]+off-tp + + ld t1,(sp) + ld t2,8(sp) + add sp,sp,16 + jr t0 + ``` ```sh (mkdir -p out/rv64 && cd out/rv64 && ../../configure --target=3Driscv64-linux-gnu && make -j 50) ``` Adjust `~/musl/out/rv64/lib/musl-gcc.specs` and update `~/musl/out/rv64/obj/musl-gcc` ```sh cat > ~/musl/out/rv64/obj/musl-gcc < ./a.c < int foo(); int bar(); int main() { assert(foo() =3D=3D 2); assert(foo() =3D=3D 4); assert(bar() =3D=3D 2); assert(bar() =3D=3D 4); } eof cat > ./b.c < __thread int tls0; extern __thread int tls1; int foo() { return ++tls0 + ++tls1; } static __thread int tls2, tls3; int bar() { return ++tls2 + ++tls3; } eof echo '__thread int tls1;' > ./c.c sed 's/ /\t/' > ./Makefile <<'eof' .MAKE.MODE =3D meta curDirOk=3Dtrue CC :=3D ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld LDFLAGS :=3D -Wl,-rpath=3D. all: a0 a1 a2 run: all ./a0 && ./a1 && ./a2 b.o: bb.s /tmp/Rel/bin/clang --target=3Driscv64-linux -c bb.s -o $@ c.so: c.o; ${LINK.c} -shared $> -o $@ bc.so: b.o c.o; ${LINK.c} -shared $> -o $@ b.so: b.o c.so; ${LINK.c} -shared $> -o $@ a0: a.o b.o c.o; ${LINK.c} $> -o $@ a1: a.o b.so; ${LINK.c} $> -o $@ a2: a.o bc.so; ${LINK.c} $> -o $@ eof ``` Compile b.c to bb.s. Replace general dynamic code sequences (e.g. `la.tls.gd a0,tls0; call __tls_get_addr@plt`) with TLSDESC, e.g. ``` .Ltlsdesc_hi0: auipc a0, %tlsdesc_hi(tls0) ld a1, %tlsdesc_load_lo(.Ltlsdesc_hi0)(a0) addi a0, a0, %tlsdesc_add_lo(.Ltlsdesc_hi0) jalr t0, 0(a1), %tlsdesc_call(.Ltlsdesc_hi0) add a0, a0, tp ``` Apply LLVM CodeGen/assembly and lld patch (https://github.com/llvm/llvm-project/pull/79099) to llvm-project. Build clang and lld. Create an alias `bin/ld.lld` to be used with `-Bbin -fuse-ld=3Dlld`. `bmake run` =3D> succeeded! % bmake run ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -c a.c /tmp/Rel/bin/clang --target=3Driscv64-linux -c bb.s -o b.o ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -c c.c ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. a.o b.o c.o -o a0 ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. -shared c.o -o c.so ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. -shared b.o c.so -o b.so ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. a.o b.so -o a1 ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. -shared b.o c.o -o bc.so ~/musl/out/rv64/obj/musl-gcc -O1 -g -fpic -Bbin -fuse-ld=3Dlld -g -Wl,-rpath=3D. a.o bc.so -o a2 ./a0 && ./a1 && ./a2