From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id 9EA3227244 for ; Tue, 13 Feb 2024 15:48:12 +0100 (CET) Received: (qmail 29794 invoked by uid 550); 13 Feb 2024 14:45:06 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 29722 invoked from network); 13 Feb 2024 14:45:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707835674; x=1708440474; darn=lists.openwall.com; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=oqbZPuHuWxNBE11H3FWpMVJUZLUnRRzOS3uMZLM7vBU=; b=Nu1FQ1RxlKmvDPhVwumPgdr1Wr9T58DrJ0Rn6tWgrzdcbFnGRhnTg4oTSvl0FNfBYU vlJ/NcVlk4BDs4BuSsqqKcZLVmPXWB/v/KMFhQcr00DNLYAcn50uK8s3wDgk46oGnJ7c 0qRTEdTMO6N1q4Boe3W7mpBpNG23TxmZH8EDfcq1R6isKFlms361w9CcWYClk9NLJRjA r2kvKKlSnG8WzHHO2N9Self3v1XpcWdv3XiRL7ZqTGRLhZwmpUvmvl411HncYgBdMWwG 1KTDIfyDz710bI9zrTyQntLKmTwf2YjXCYjFCMdoFplJXQUb9UmeybpWk0UinGKnSNkU o7Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707835674; x=1708440474; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oqbZPuHuWxNBE11H3FWpMVJUZLUnRRzOS3uMZLM7vBU=; b=ndjRg7zqfqldHnZHyFaeMMK/UQlNJitXB03H2hfSkYbmQvoCN07VZKSkAfB2TonkJq rjb7940L6qX3hlCZ9fnSuwYmDr0BQQOYASqI5nJvb9LPzcfW/sckpz26M1fiMtaGu7+r 2wUy4D09+x2qxoeyKtx4Nie1f+JYi7rOWoB2rFRtrY7++ZpptCXkV/HOeDZsq7znPdk0 7//mD4e5iUVLkaCUSm/KweIS2eC9CZXBqA/jm+D+KWDmSBDIPbKigh7iPdAJdJb1OLYq 5Mij4JSvWXRBCmqosuWg3BsBpOBYv9e7lkYwenssCfJ0TRY2DAlrlYuIEEOlXdLvX+oV 6iMQ== X-Forwarded-Encrypted: i=1; AJvYcCXzGkwvNEfBKaGHja56AbdP0YLWdAKWZHiy7PhGNgcZU69J3OsEZeUDWgvEZ5tCbZkWmkpn5a6l5oGkFlgQgvTmlK+SKOncsA== X-Gm-Message-State: AOJu0YzqBCcdSeePHH3MC2BdyyrZJ/4IuxSXwgSDyqtH0uz2ug/ppcBm usk0NrEfmhw1tkHQbk+q6H+oYo6bxuG/9lI3Ij5CvXSWsaoTRc1pdkTSNYMeoT8PfOIyjgQJRv6 7d7g6YyvAEAlSUFhD8MAf2w5PaDU= X-Google-Smtp-Source: AGHT+IG6RUi1uRjtlvF4Dn27SFZ0TP2JW9wXipYXS9BZ6dOwI2D+Tnvfm9As3TKtAZNoKWo8D2qT89EHxB+AkeLAbfI= X-Received: by 2002:a05:6512:3196:b0:511:737d:661a with SMTP id i22-20020a056512319600b00511737d661amr1108733lfe.28.1707835673913; Tue, 13 Feb 2024 06:47:53 -0800 (PST) MIME-Version: 1.0 References: <20240212184236.GZ4163@brightrain.aerifal.cx> <20240212224657.GA4163@brightrain.aerifal.cx> <20240213020834.GB4163@brightrain.aerifal.cx> In-Reply-To: <20240213020834.GB4163@brightrain.aerifal.cx> From: William Roberts Date: Tue, 13 Feb 2024 08:47:42 -0600 Message-ID: To: Rich Felker Cc: enh , musl@lists.openwall.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [musl] PAC/BTI Support on aarch64 On Mon, Feb 12, 2024 at 8:08=E2=80=AFPM Rich Felker wrote= : > > On Mon, Feb 12, 2024 at 05:18:22PM -0600, William Roberts wrote: > > On Mon, Feb 12, 2024 at 5:05=E2=80=AFPM enh wrote: > > > > > > On Mon, Feb 12, 2024 at 2:46=E2=80=AFPM Rich Felker = wrote: > > > > > > > > On Mon, Feb 12, 2024 at 03:25:48PM -0600, William Roberts wrote: > > > > > On Mon, Feb 12, 2024 at 12:42=E2=80=AFPM Rich Felker wrote: > > > > > > > > > > > > On Mon, Feb 12, 2024 at 10:38:50AM -0600, William Roberts wrote= : > > > > > > > Hello, > > > > > > > > > > > > > > I was just wondering if there was any work being done to supp= ort PAC > > > > > > > and BTI in aarch64? I could add support but didn't want to du= plicate > > > > > > > the work. > > > > > > > > > > > > I'm not aware of any active work on this, but before writing a = full > > > > > > implementation, it would be really helpful to start with a basi= c > > > > > > proposal for the scope of changes needed to make it work to ass= ess > > > > > > whether these are manageable and acceptable cost. > > > > > > > > > > It's a matter of building with -mbranch-protection=3Dstandard > > > > > > > > > > Just the ASM labels need the first instruction to be a BTI. They'= re in > > > > > the NOP space > > > > > so they are backwards compatible, older hardware will just NOP it= . > > > > > > > > I think it's a little more elaborate than that. Those asm instructi= ons > > > > need to be added (probably as .instr or .word or something, unless > > > > there's a way to spell this particular nop that existing tooling wi= ll > > > > understand). > > > > > > depends on your toolchain version. when we added this to bionic, the > > > toolchain work was still happening. so you'll want to test against > > > whatever your oldest-supported toolchain is. > > > > > > > You just use the hint instructions, they are understood by = old > > toolchains. But you can only support a subset of the BTI/PAC instructio= ns > > but it's been enough for most projects that follow the normal ABI conve= ntions > > like OpenSSL/BoringSSL,etc, but not enough for libffi for example. > > If hint goes all the way back, that's probably fine and ideal to use. It should. Is there a known minimal tool chain requirement and I can test? > > > > > Or it could be made conditional, but that would require > > > > converting any asm that's not already .S files to .S. Not bad, but = not > > > > as in inline asm? Unless it's a branch target, no need. > > No, .S (preprocessed) vs .s (not). But if the hint insn works, I think > just having it there unconditionally is probably the way to go. > > > > > I also wondered if [sig]setjmp/longjmp would be affected, but proba= bly > > > > not. > > > > > > bionic does use PAC, but i think glibc has its own "pointer mangling"= thing? > > > > You need it, as the first instruction from a branch (where longjmp retu= rns to) > > needs to be a BTI instruction. > > Is that different from a normal function return? No, anywhere branches are allowed, a BTI instruction must be the first instruction. BTI is just a way for software to say, hey this is a valid jump/branch target, allow it. This reduces the amount of gadgets available to an attacker, which is why libc is such a juicy target, as it's in everything. A lot of things static link it, which effectively turns it off for the whole process. > > Note that in the case of sigsetjmp, (sig)longjmp returns to a point > inside the sigsetjmp asm, so that point needs the annotation I think. > > > > > > It's been done for many projects, glibc and bionic have it. The > > > > > problem with BTI is that when one item in the link > > > > > list doesn't support BTI the loader/linker turns it off. So when = it's > > > > > something like a libc that is fundamental in the link chain, > > > > > it turns it off for everything. > > > > > > > > This presumably requires some kind of machinery for how dynamic > > > > linking will work, and possibly turning it off if a library without= it > > > > is dlopened? > > > > > > > > My understanding doing some brief searches though was that you can > > > > individually mprotect it off in certain regions. So maybe it's > > > > possible to just enable only for DSOs that support it? > > > > > > correct. > > OK, that's good to know. So which direction is it? Do DSOs that > support BTI need it explicitly turned on via mprotect/mmap flags? Yes, so the kernel will manage the EL1 register flag for this, and then mprotect sets the PROT_BTI flag during dlopen(). > Or > is there some process-global flag to turn it on, and then ones that > don't support it need it turned off? EL1 MSR register (I forget which one offhand), but the granularity is managed at the page level. > > I suspect it's possible to first enable BTI for third-party libraries > as a feature of the dynamic linker, If you mean, check the GNU Notes section for BTI enabled and set PROT_BTI via mprotect, that's just one of the many patches, but can be taken independently. > and add BTI support for libc > itself as a separate thing. That might be a nice factoring to make > changes minimal and easy for ppl to read. This is just a matter of organizing things, there's no dependency between enabling the linker and enabling the library itself. So of course that shou= ldn't come as one giant patch. It's important to note, that even when enabling the assembly code files, if= the C level source is not built with -mbranch-protection=3Dstandard, the featur= e will remain off for the library. BTI is enabled for third party packages on Fedora by default: - https://fedoraproject.org/wiki/Changes/Aarch64_PointerAuthentication The problem is, now all the packages that don't use the default set of CFLAGS and/or roll their own asm. > > The changes in dynlink.c should be as arch-agnostic as possible. If > there's a corresponding feature on other archs I can't think of anything like this offhand, but aarches may want to add pr= ot flags to mprotect calls. > it should use the same > basic code, with arch-specific headers (arch/$ARCH/reloc.h) defining > the mechanisms for evaluating if an ELF file is compatible, how to do > the mprotect, etc. it usually #ifdef aarch64 if (gnu_notes_bti_set && (prot & PROT_EXEC)) { prot |=3D PROT_BTI; else { prot &=3D ~PROT_BTI; } #endif mprotect(..., prot); but this could be done with something like an arch specific macro fn or inline in a header that just does nothing for most architectures or a weak symbol, but I am always worried with weak symbols someone might override it in a bad way. > > > > > > The initial scope of code changes would be what's reported when > > > > > LDFLAGS=3D-Wl,-zforce-bti,--fatal-warnings > > > > > > > > Is there a way to disable these warnings so that every asm file doe= s > > > > not need to be cluttered with annotations? > > > > > > well, that's the ELF note stuff i was talking about, and if you don't > > > have it you'll fall foul of the static linker saying "not all this > > > code is BTI-enabled, therefore this .so isn't", and the dynamic linke= r > > > doing nothing because the static linker effectively tells it not to. > > > > Yep, well said ENH. It's been since Android since we crossed paths :-). > > > > It's not that hard to annotate an asm file :-p I forget what project > > (I think it was gnutls, but they just use openssl's code for the asm) > > but I just put it in a header file and by virtue of #include'ing it you= get the > > notes added. > > Yes, we generally don't do that. There are no "asm headers" in musl; > all asm files are self-contained and readable standalone. So if > there's no way to tell the assembler/linker from the command line that > files are BTI-compatible without generating a huge load of warning > spam, I guess it's a mess of copy-and-paste... > > Rich