From: Szabolcs Nagy <nsz@port70.net>
To: musl@lists.openwall.com
Subject: Re: TLS issue on aarch64
Date: Fri, 25 May 2018 16:50:59 +0200 [thread overview]
Message-ID: <20180525145059.GG4418@port70.net> (raw)
In-Reply-To: <CA+RmVGxmz2HTos2uPYW6PetrjFGZgWejW2aqkx7-9MjkF_H9dg@mail.gmail.com>
* Phillip Berndt <phillip.berndt@googlemail.com> [2018-05-25 14:40:14 +0200]:
> I'm experiencing a TLS-related error with musl on aarch64. This is my
> test program:
>
> ----8<--------------
> #include <stdio.h>
>
> __thread int foo = 1234;
> __thread int bar __attribute__((aligned(0x100))) = 5678;
>
> int main(int argc, char **argv)
> {
> printf("0x%p: %d\n", &foo, foo);
> printf("0x%p: %d\n", &bar, bar);
>
> return 0;
> }
> ---->8---------------
>
> I'm compiling this into a static binary with -O2 -static. With glibc and
> gcc 5.4.0 (tried with 7.2 as well), this gives me the expected output. With
> musl libc 1.1.19, I instead see
>
> ----8<--------------
> 0x0x7fa7adf2f0: 0
> 0x0x7fa7adf3f0: 0
> ---->8---------------
>
> Note that this is the wrong address (not aligned), and that the memory has
> unexpected content as well.
>
> I did some initial debugging, but now I'm stuck and need some help. What I've
> found so far:
>
> * GCC apparently emits code that expects the tpidr_el0 register to contain a
> pointer to the TLS memory, and it expects that the loader unconditionally
> offsets the first variable by the TLS alignment into said memory:
>
> Disassembly of the code that loads &foo:
> ----8<--------------
> 4001a4: d53bd053 mrs x19, tpidr_el0
> 4001a8: 91400273 add x19, x19, #0x0, lsl #12
> 4001ac: 91040273 add x19, x19, #0x100
> ----8<--------------
>
> (If I align the variable by 0x1000 instead then the code changes
> acoordingly.)
>
> * Musl, on the other hand, in __copy_tls, initializes tpidr_el0 with a
> pointer 16 bytes from the end of struct pthread, and copies the TLS
> initializer code directly behind that struct, without adding extra
> padding.
>
yeah i think on aarch64 (which is TLS_ABOVE_TP) musl expects a layout
-------------------------- ----------- - -
| struct pthread | 16 | tlsvar
-------------------------- ----------- - -
^ ^ <----->
self tp tls offset
(aligned)
while it should be
-------------------------- ----------- - -
| struct pthread | 16 | tlsvar
-------------------------- ----------- - -
^ ^<-------------->
self tp tls offset
(aligned)
i think the constraints for tp are:
- tp must be aligned to 'tls_align'
- tp must be at a small fixed offset from the end
of pthread struct (so asm code can access the dtv)
- tp + off must be usable memory for tls for off >= 16
(this is aarch64 specific)
i'm not yet sure what's the best fix.
> Hence the code tries to access the TLS variables at the wrong location.
>
> The following patch fixes the issue, but only if musl is then compiled with
> optimizations off. With optimizations, the compiler emits the *same* code for
> both variants. Also, the patch probably has some unexpected side-effects, too -
> I'm just adding it here as a starting point for further debugging.
>
> Any help is greatly appreciated :-)
>
> - Phillip
>
> ----
>
> diff --git a/arch/aarch64/pthread_arch.h b/arch/aarch64/pthread_arch.h
> index b2e2d8f..c69f6f1 100644
> --- a/arch/aarch64/pthread_arch.h
> +++ b/arch/aarch64/pthread_arch.h
> @@ -2,10 +2,10 @@ static inline struct pthread *__pthread_self()
> {
> char *self;
> __asm__ __volatile__ ("mrs %0,tpidr_el0" : "=r"(self));
> - return (void*)(self + 16 - sizeof(struct pthread));
> + return (void*)(self - sizeof(struct pthread));
> }
>
> #define TLS_ABOVE_TP
> -#define TP_ADJ(p) ((char *)(p) + sizeof(struct pthread) - 16)
> +#define TP_ADJ(p) ((char *)(p) + sizeof(struct pthread))
>
this is ok, but wastes 16 bytes after the pthread struct
where will be no tls data (and i guess the allocated tls
size should be adjusted accordingly as well as tlsdesc.s).
> #define MC_PC pc
> diff --git a/src/env/__init_tls.c b/src/env/__init_tls.c
> index b125eb1..3a3c307 100644
> --- a/src/env/__init_tls.c
> +++ b/src/env/__init_tls.c
> @@ -42,7 +42,7 @@ void *__copy_tls(unsigned char *mem)
>
> mem += -((uintptr_t)mem + sizeof(struct pthread)) & (libc.tls_align-1);
> td = (pthread_t)mem;
> - mem += sizeof(struct pthread);
> + mem += sizeof(struct pthread) + libc.tls_align;
>
i don't think this works with the above TP_ADJ, but
yes something has to be done differently here to make
aarch64 happy.
> for (i=1, p=libc.tls_head; p; i++, p=p->next) {
> dtv[i] = mem + p->offset;
next prev parent reply other threads:[~2018-05-25 14:50 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-25 12:40 Phillip Berndt
2018-05-25 14:50 ` Szabolcs Nagy [this message]
2018-05-25 21:38 ` Szabolcs Nagy
2018-05-25 22:20 ` Phillip Berndt
2018-05-26 0:54 ` Szabolcs Nagy
2018-05-26 8:24 ` Phillip Berndt
2018-05-27 0:34 ` Rich Felker
2018-05-28 20:47 ` Szabolcs Nagy
2018-05-28 22:15 ` Rich Felker
2018-05-29 6:33 ` Szabolcs Nagy
2018-05-31 15:22 ` Phillip Berndt
2018-06-01 0:11 ` Szabolcs Nagy
2018-06-01 0:52 ` Rich Felker
2018-06-01 9:38 ` Szabolcs Nagy
2018-05-27 0:17 ` Rich Felker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180525145059.GG4418@port70.net \
--to=nsz@port70.net \
--cc=musl@lists.openwall.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).