From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.1 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id 058222744B for ; Tue, 13 Feb 2024 03:08:32 +0100 (CET) Received: (qmail 7363 invoked by uid 550); 13 Feb 2024 02:05:31 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 7325 invoked from network); 13 Feb 2024 02:05:31 -0000 Date: Mon, 12 Feb 2024 21:08:34 -0500 From: Rich Felker To: William Roberts Cc: enh , musl@lists.openwall.com Message-ID: <20240213020834.GB4163@brightrain.aerifal.cx> References: <20240212184236.GZ4163@brightrain.aerifal.cx> <20240212224657.GA4163@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] PAC/BTI Support on aarch64 On Mon, Feb 12, 2024 at 05:18:22PM -0600, William Roberts wrote: > On Mon, Feb 12, 2024 at 5:05 PM enh wrote: > > > > On Mon, Feb 12, 2024 at 2:46 PM Rich Felker wrote: > > > > > > On Mon, Feb 12, 2024 at 03:25:48PM -0600, William Roberts wrote: > > > > On Mon, Feb 12, 2024 at 12:42 PM Rich Felker wrote: > > > > > > > > > > On Mon, Feb 12, 2024 at 10:38:50AM -0600, William Roberts wrote: > > > > > > Hello, > > > > > > > > > > > > I was just wondering if there was any work being done to support PAC > > > > > > and BTI in aarch64? I could add support but didn't want to duplicate > > > > > > the work. > > > > > > > > > > I'm not aware of any active work on this, but before writing a full > > > > > implementation, it would be really helpful to start with a basic > > > > > proposal for the scope of changes needed to make it work to assess > > > > > whether these are manageable and acceptable cost. > > > > > > > > It's a matter of building with -mbranch-protection=standard > > > > > > > > Just the ASM labels need the first instruction to be a BTI. They're in > > > > the NOP space > > > > so they are backwards compatible, older hardware will just NOP it. > > > > > > I think it's a little more elaborate than that. Those asm instructions > > > need to be added (probably as .instr or .word or something, unless > > > there's a way to spell this particular nop that existing tooling will > > > understand). > > > > depends on your toolchain version. when we added this to bionic, the > > toolchain work was still happening. so you'll want to test against > > whatever your oldest-supported toolchain is. > > > > You just use the hint instructions, they are understood by old > toolchains. But you can only support a subset of the BTI/PAC instructions > but it's been enough for most projects that follow the normal ABI conventions > like OpenSSL/BoringSSL,etc, but not enough for libffi for example. If hint goes all the way back, that's probably fine and ideal to use. > > > Or it could be made conditional, but that would require > > > converting any asm that's not already .S files to .S. Not bad, but not > > as in inline asm? Unless it's a branch target, no need. No, .S (preprocessed) vs .s (not). But if the hint insn works, I think just having it there unconditionally is probably the way to go. > > > I also wondered if [sig]setjmp/longjmp would be affected, but probably > > > not. > > > > bionic does use PAC, but i think glibc has its own "pointer mangling" thing? > > You need it, as the first instruction from a branch (where longjmp returns to) > needs to be a BTI instruction. Is that different from a normal function return? Note that in the case of sigsetjmp, (sig)longjmp returns to a point inside the sigsetjmp asm, so that point needs the annotation I think. > > > > It's been done for many projects, glibc and bionic have it. The > > > > problem with BTI is that when one item in the link > > > > list doesn't support BTI the loader/linker turns it off. So when it's > > > > something like a libc that is fundamental in the link chain, > > > > it turns it off for everything. > > > > > > This presumably requires some kind of machinery for how dynamic > > > linking will work, and possibly turning it off if a library without it > > > is dlopened? > > > > > > My understanding doing some brief searches though was that you can > > > individually mprotect it off in certain regions. So maybe it's > > > possible to just enable only for DSOs that support it? > > > > correct. OK, that's good to know. So which direction is it? Do DSOs that support BTI need it explicitly turned on via mprotect/mmap flags? Or is there some process-global flag to turn it on, and then ones that don't support it need it turned off? I suspect it's possible to first enable BTI for third-party libraries as a feature of the dynamic linker, and add BTI support for libc itself as a separate thing. That might be a nice factoring to make changes minimal and easy for ppl to read. The changes in dynlink.c should be as arch-agnostic as possible. If there's a corresponding feature on other archs, it should use the same basic code, with arch-specific headers (arch/$ARCH/reloc.h) defining the mechanisms for evaluating if an ELF file is compatible, how to do the mprotect, etc. > > > > The initial scope of code changes would be what's reported when > > > > LDFLAGS=-Wl,-zforce-bti,--fatal-warnings > > > > > > Is there a way to disable these warnings so that every asm file does > > > not need to be cluttered with annotations? > > > > well, that's the ELF note stuff i was talking about, and if you don't > > have it you'll fall foul of the static linker saying "not all this > > code is BTI-enabled, therefore this .so isn't", and the dynamic linker > > doing nothing because the static linker effectively tells it not to. > > Yep, well said ENH. It's been since Android since we crossed paths :-). > > It's not that hard to annotate an asm file :-p I forget what project > (I think it was gnutls, but they just use openssl's code for the asm) > but I just put it in a header file and by virtue of #include'ing it you get the > notes added. Yes, we generally don't do that. There are no "asm headers" in musl; all asm files are self-contained and readable standalone. So if there's no way to tell the assembler/linker from the command line that files are BTI-compatible without generating a huge load of warning spam, I guess it's a mess of copy-and-paste... Rich