[musl] [PATCH 0/3] RFC: Enable PAC and BTI for aarch64

[musl] [PATCH 0/3] RFC: Enable PAC and BTI for aarch64

[Bill Roberts]

Hello,

I am floating a proposal to add PAC (Pointer Authentication) and BTI
(Branch Target Identification) support for MUSL on AArch64 targets.

For anyone looking for a quick overview of what this means for software
developers, this blog post provides a good summary:
  https://developer.arm.com/community/arm-community-blogs/b/architectures-
  and-processors-blog/posts/enabling-pac-and-bti-on-aarch64

(Full disclosure: I may be a bit biased there.)

Summary below, but the git commit messages and comments should clarify
anything not immediately obvious (hopefully).

MUSL is fortunately well-structured, so adding this support is not too
complex. I wrote the assembly assuming the hardware supports PAC and BTI,
and then used awk to post-process it out when needed. This approach seemed
simpler than tracking stack usage, identifying jump vs. call targets, and
trying to add the features afterward within awk.
When writing assembly, it is easier to assume the feature is supported.
By doing so, it keeps the relevant instructions visible rather than
debugging an awk script that might surprise you or be inserting extra
instructions unexpectedly.

For BTI, there are two aspects to consider. The first is the loader, which
needs to mmap/mprotect with PROT_BTI. The second is the addition of BTI
instructions and GNU note annotations. These two parts are independent of
each other.

For PAC, it is mainly a matter of enabling the instructions and adding the
GNU Notes annotations.

-- Possible Alternative Implementation --

Another way this could be done is to add the instructions unconditionally.
i.e do not remove them. One nice property of these extensions is that binaries
can include PAC and BTI instructions unconditionally. On hardware that does
not support these features, the hint-space instructions effectively NOP,
maintaining ABI and backward compatibility. This means we do not need to build separate
variants or introduce conditional support in most cases. The only thing
to consider would be switching PAC keys from A to B.


Thanks,
Bill

Bill Roberts (3):
  dso/aarch64: add PROT_BTI support for mem maps
  Makefile: support awk processing of .S files
  aarch64: enable PAC and BTI instruction support in musl build

 Makefile                          |  25 ++++--
 arch/aarch64/crt_arch.h           |   3 +
 configure                         |  14 +++-
 crt/aarch64/crti.s                |   2 +
 crt/aarch64/crtn.s                |   2 +
 include/elf.h                     |   2 +
 ldso/dynlink.c                    |  50 ++++++++++--
 src/fenv/aarch64/fenv.s           |   7 ++
 src/ldso/aarch64/arch_ldso_hook.c |  90 ++++++++++++++++++++++
 src/ldso/aarch64/arch_ldso_hook.h |  21 +++++
 src/ldso/aarch64/tlsdesc.s        |   2 +
 src/process/aarch64/vfork.s       |   1 +
 src/setjmp/aarch64/longjmp.s      |   1 +
 src/setjmp/aarch64/setjmp.s       |   1 +
 src/signal/aarch64/restore.s      |   1 +
 src/signal/aarch64/sigsetjmp.s    |   2 +
 src/string/aarch64/memcpy.S       |   1 +
 src/string/aarch64/memset.S       |   1 +
 src/thread/aarch64/__unmapself.s  |   1 +
 src/thread/aarch64/clone.s        |   1 +
 src/thread/aarch64/syscall_cp.s   |   1 +
 tools/pac-bti-aarch64.awk         | 122 ++++++++++++++++++++++++++++++
 22 files changed, 336 insertions(+), 15 deletions(-)
 create mode 100644 src/ldso/aarch64/arch_ldso_hook.c
 create mode 100644 src/ldso/aarch64/arch_ldso_hook.h
 create mode 100644 tools/pac-bti-aarch64.awk

-- 
2.51.0

[musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Bill Roberts]

This change adds support for setting PROT_BTI on executable mappings
when the corresponding ELF object includes the GNU_PROPERTY_AARCH64_FEATURE_1_BTI
note. This ensures correct Branch Target Identification (BTI) enforcement
as defined in the Arm Architecture Reference Manual (Arm ARM) and the
ELF for the Arm 64-bit Architecture (AArch64) ABI supplement.

When loading shared objects or executables, the dynamic loader now checks
for BTI-related GNU notes and applies the appropriate protection flags
via `mprotect` and `mmap` during mapping. This aligns musl’s behavior with
other modern loaders, improving security and correctness on systems with BTI
enabled hardware.

References:
 - Arm Architecture Reference Manual for A-profile architecture (Arm ARM):
   https://developer.arm.com/documentation/ddi0487/latest
 - ELF for the Arm® 64-bit Architecture (AArch64) ABI supplement:
   https://github.com/ARM-software/abi-aa/blob/main/aaelf64/aaelf64.rst
 - Arm Community Blog – *Enabling Pointer Authentication and Branch Target Identification*:
   1. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part1
   2. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part2
   3. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part3

Signed-off-by: Bill Roberts <bill.roberts@arm.com>
---
 Makefile                          |  2 +-
 configure                         |  1 +
 include/elf.h                     |  2 +
 ldso/dynlink.c                    | 50 ++++++++++++++---
 src/ldso/aarch64/arch_ldso_hook.c | 90 +++++++++++++++++++++++++++++++
 src/ldso/aarch64/arch_ldso_hook.h | 21 ++++++++
 6 files changed, 158 insertions(+), 8 deletions(-)
 create mode 100644 src/ldso/aarch64/arch_ldso_hook.c
 create mode 100644 src/ldso/aarch64/arch_ldso_hook.h

diff --git a/Makefile b/Makefile
index 3ad88b35..a20bafaf 100644
--- a/Makefile
+++ b/Makefile
@@ -47,7 +47,7 @@ CFLAGS_AUTO = -Os -pipe
 CFLAGS_C99FSE = -std=c99 -ffreestanding -nostdinc 
 
 CFLAGS_ALL = $(CFLAGS_C99FSE)
-CFLAGS_ALL += -D_XOPEN_SOURCE=700 -I$(srcdir)/arch/$(ARCH) -I$(srcdir)/arch/generic -Iobj/src/internal -I$(srcdir)/src/include -I$(srcdir)/src/internal -Iobj/include -I$(srcdir)/include
+CFLAGS_ALL += -D_XOPEN_SOURCE=700 -I$(srcdir)/arch/$(ARCH) -I$(srcdir)/arch/generic -Iobj/src/internal -I$(srcdir)/src/include -I$(srcdir)/src/internal -Iobj/include -I$(srcdir)/include -I$(srcdir)/src/ldso/$(ARCH)
 CFLAGS_ALL += $(CPPFLAGS) $(CFLAGS_AUTO) $(CFLAGS)
 
 LDFLAGS_ALL = $(LDFLAGS_AUTO) $(LDFLAGS)
diff --git a/configure b/configure
index bc9fbe48..3809f24c 100755
--- a/configure
+++ b/configure
@@ -671,6 +671,7 @@ fi
 
 if test "$ARCH" = "aarch64" ; then
 trycppif __AARCH64EB__ "$t" && SUBARCH=${SUBARCH}_be
+CFLAGS_AUTO="${CFLAGS_AUTO} -DARCH_SUPPORTS_DL_ADD_PROTECTIONS"
 fi
 
 if test "$ARCH" = "loongarch64" ; then
diff --git a/include/elf.h b/include/elf.h
index d6ae539a..e40b815d 100644
--- a/include/elf.h
+++ b/include/elf.h
@@ -1110,7 +1110,9 @@ typedef struct {
 #define NT_GNU_GOLD_VERSION	4
 #define NT_GNU_PROPERTY_TYPE_0	5
 
+#define GNU_PROPERTY_AARCH64_FEATURE_1_AND 0xc0000000
 
+#define GNU_PROPERTY_AARCH64_FEATURE_1_BTI (1U << 0)
 
 typedef struct {
   Elf32_Xword m_value;
diff --git a/ldso/dynlink.c b/ldso/dynlink.c
index 715948f4..d204f2d5 100644
--- a/ldso/dynlink.c
+++ b/ldso/dynlink.c
@@ -682,6 +682,21 @@ static void unmap_library(struct dso *dso)
 	}
 }
 
+#ifdef ARCH_SUPPORTS_DL_ADD_PROTECTIONS
+#include <arch_ldso_hook.h>
+#else
+static inline unsigned dl_add_protections(const Elf64_Phdr *ph,
+	const Elf64_Ehdr *eh, int fd, unsigned *prot, unsigned *mask)
+{
+	(void)ph;
+	(void)eh;
+	(void)fd;
+	(void)prot;
+	(void)mask;
+	return 0;
+}
+#endif
+
 static void *map_library(int fd, struct dso *dso)
 {
 	Ehdr buf[(896+sizeof(Ehdr))/sizeof(Ehdr)];
@@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
 	off_t off_start;
 	Ehdr *eh;
 	Phdr *ph, *ph0;
-	unsigned prot;
+	unsigned prot, arch_prot, arch_mask;
 	unsigned char *map=MAP_FAILED, *base;
 	size_t dyn=0;
 	size_t tls_image=0;
@@ -720,6 +735,16 @@ static void *map_library(int fd, struct dso *dso)
 	} else {
 		ph = ph0 = (void *)((char *)buf + eh->e_phoff);
 	}
+
+	/*
+	 * some architectures may have additional protection flags embedded in the
+	 * ELF section somewhere. Start with those and OR in the extras. Do this
+	 * before we need to map any sections.
+	 */
+	if(dl_add_protections(ph, eh, fd, &arch_prot, &arch_mask))
+		goto error;
+
+
 	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
 		if (ph->p_type == PT_DYNAMIC) {
 			dyn = ph->p_vaddr;
@@ -746,6 +771,7 @@ static void *map_library(int fd, struct dso *dso)
 			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
 				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
 				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
+			prot |= prot&arch_mask ? arch_prot : 0;
 		}
 		if (ph->p_vaddr+ph->p_memsz > addr_max) {
 			addr_max = ph->p_vaddr+ph->p_memsz;
@@ -762,6 +788,7 @@ static void *map_library(int fd, struct dso *dso)
 			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
 				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
 				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
+			prot |= prot&arch_mask ? arch_prot : 0;
 			map = mmap(0, ph->p_memsz + (ph->p_vaddr & PAGE_SIZE-1),
 				prot, MAP_PRIVATE,
 				fd, ph->p_offset & -PAGE_SIZE);
@@ -780,6 +807,7 @@ static void *map_library(int fd, struct dso *dso)
 				size_t pgbrk = brk + PAGE_SIZE-1 & -PAGE_SIZE;
 				size_t pgend = brk + ph->p_memsz - ph->p_filesz
 					+ PAGE_SIZE-1 & -PAGE_SIZE;
+				/* arch_prot has already been added */
 				if (pgend > pgbrk && mmap_fixed(map+pgbrk,
 					pgend-pgbrk, prot,
 					MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,
@@ -801,11 +829,16 @@ static void *map_library(int fd, struct dso *dso)
 	 * the length of the file. This is okay because we will not
 	 * use the invalid part; we just need to reserve the right
 	 * amount of virtual address space to map over later. */
-	map = DL_NOMMU_SUPPORT
-		? mmap((void *)addr_min, map_len, PROT_READ|PROT_WRITE|PROT_EXEC,
-			MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
-		: mmap((void *)addr_min, map_len, prot,
-			MAP_PRIVATE, fd, off_start);
+	if (map == DL_NOMMU_SUPPORT) {
+		prot = PROT_READ|PROT_WRITE|PROT_EXEC;
+		prot |= prot&arch_mask ? arch_prot : 0;
+		map = mmap((void *)addr_min, map_len, prot,
+				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
+	} else {
+		prot = prot & arch_mask ? prot | arch_prot : prot;
+		map = mmap((void *)addr_min, map_len, prot,
+				MAP_PRIVATE, fd, off_start);
+	}
 	if (map==MAP_FAILED) goto error;
 	dso->map = map;
 	dso->map_len = map_len;
@@ -835,6 +868,7 @@ static void *map_library(int fd, struct dso *dso)
 		prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
 			((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
 			((ph->p_flags&PF_X) ? PROT_EXEC : 0));
+		prot |= prot&arch_mask ? arch_prot : 0;
 		/* Reuse the existing mapping for the lowest-address LOAD */
 		if ((ph->p_vaddr & -PAGE_SIZE) != addr_min || DL_NOMMU_SUPPORT)
 			if (mmap_fixed(base+this_min, this_max-this_min, prot, MAP_PRIVATE|MAP_FIXED, fd, off_start) == MAP_FAILED)
@@ -849,7 +883,9 @@ static void *map_library(int fd, struct dso *dso)
 	}
 	for (i=0; ((size_t *)(base+dyn))[i]; i+=2)
 		if (((size_t *)(base+dyn))[i]==DT_TEXTREL) {
-			if (mprotect(map, map_len, PROT_READ|PROT_WRITE|PROT_EXEC)
+			prot = PROT_READ|PROT_WRITE|PROT_EXEC;
+			prot |= prot&arch_mask ? arch_prot : 0;
+			if (mprotect(map, map_len, prot)
 			    && errno != ENOSYS)
 				goto error;
 			break;
diff --git a/src/ldso/aarch64/arch_ldso_hook.c b/src/ldso/aarch64/arch_ldso_hook.c
new file mode 100644
index 00000000..9a755707
--- /dev/null
+++ b/src/ldso/aarch64/arch_ldso_hook.c
@@ -0,0 +1,90 @@
+#include <elf.h>
+#include <sys/mman.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+unsigned dl_add_protections(const Elf64_Phdr *ph, const Elf64_Ehdr *eh, int fd,
+		unsigned *prot, unsigned *mask) {
+	size_t i;
+	char *buf = NULL;
+	unsigned rc = 1;
+
+	/*
+	 * Unfortunately, we need to loop through the ELF header, as we need
+	 * to get the PROT flags before we map anything in and they could be
+	 * anywhere.
+	 */
+	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
+
+		/* skip non note sections */
+		if (ph->p_type != PT_NOTE)
+			continue;
+
+		buf = malloc(ph->p_filesz);
+		if (!buf)
+			return rc;
+
+		if (pread(fd, buf, ph->p_filesz, ph->p_offset) != ph->p_filesz)
+			goto error;
+
+		const char *p = buf;
+		const char *end = buf + ph->p_filesz;
+
+		/* for each ELF note */
+		while (p + sizeof(Elf64_Nhdr) <= end) {
+
+			const Elf64_Nhdr *nh = (const Elf64_Nhdr*) p;
+			p += sizeof(Elf64_Nhdr);
+
+			const char *name = (const char*) p;
+			p += (nh->n_namesz + 3) & ~3;  // 4-byte align
+
+			const char *desc = p;
+			p += (nh->n_descsz + 3) & ~3;
+
+			if (p > end)
+				break;
+
+			/* We're only interested in GNU notes and property type 0*/
+			if (nh->n_namesz != 4|| memcmp(name, "GNU", 4)
+				|| nh->n_type != NT_GNU_PROPERTY_TYPE_0) {
+				goto out;
+			}
+
+			const char *dp = desc;
+			const char *dend = desc + nh->n_descsz;
+
+			/* for each property in the property list, kind of like TLV record in series */
+			while (dp + 2 * sizeof(uint32_t) <= dend) {
+				uint32_t pr_type = *(const uint32_t*) dp;
+				dp += sizeof(uint32_t);
+				uint32_t pr_datasz = *(const uint32_t*) dp;
+				dp += sizeof(uint32_t);
+
+				if (dp + pr_datasz > dend)
+					break;
+
+				if (pr_type == GNU_PROPERTY_AARCH64_FEATURE_1_AND
+						&& pr_datasz >= 4) {
+					uint32_t features = *(const uint32_t*) dp;
+
+					if (features & GNU_PROPERTY_AARCH64_FEATURE_1_BTI) {
+						(*prot) |= PROT_BTI;
+						goto out;
+					}
+				}
+
+				dp += (pr_datasz + 3) & ~3;  // align to 4 bytes
+			}
+		}
+	}
+out:
+	/* Only add flags if the mapping will be executable */
+	*mask = PROT_EXEC;
+	rc = 0;
+error:
+	free(buf);
+	return rc;
+}
diff --git a/src/ldso/aarch64/arch_ldso_hook.h b/src/ldso/aarch64/arch_ldso_hook.h
new file mode 100644
index 00000000..11581712
--- /dev/null
+++ b/src/ldso/aarch64/arch_ldso_hook.h
@@ -0,0 +1,21 @@
+#ifndef LDSO_AARCH64_ARCH_LDSO_HOOK_H_
+#define LDSO_AARCH64_ARCH_LDSO_HOOK_H_
+
+/* Optional arch hook for the dynamic loader.
+ * If ARCH_SUPPORTS_DL_ADD_PROTECTIONS is defined, this function may
+ * examine ELF headers and file sections to determine whether to add
+ * arch-specific mmap/mprotect flags.
+ *
+ * Arguments:
+ *   ph, eh (in)  – program and ELF headers for the object being loaded.
+ *   fd     (in)  – file descriptor for the mapped object.
+ *   prot   (out) – protection flags to OR into existing flags.
+ *   mask   (out) – bitmask; new flags in *prot are ORed into existing_flags
+ *                  only if (existing_flags & *mask).
+ *
+ * Returns 0 on sucess.
+ */
+unsigned dl_add_protections(const Elf64_Phdr *ph,
+	const Elf64_Ehdr *eh, int fd, unsigned *prot, unsigned *mask);
+
+#endif
-- 
2.51.0

[musl] [PATCH 2/3] Makefile: support awk processing of .S files

[Bill Roberts]

Don't skip .S files when post-processing with awk.

Signed-off-by: Bill Roberts <bill.roberts@arm.com>
---
 Makefile | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index a20bafaf..4e62e0b3 100644
--- a/Makefile
+++ b/Makefile
@@ -136,15 +136,17 @@ CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
 # Choose invocation of assembler to be used
 ifeq ($(ADD_CFI),yes)
 	AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
+	CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
 else
 	AS_CMD = $(CC_CMD)
+	CCS_CMD = $(CC_CMD)
 endif
 
 obj/%.o: $(srcdir)/%.s
 	$(AS_CMD)
 
 obj/%.o: $(srcdir)/%.S
-	$(CC_CMD)
+	$(CCS_CMD)
 
 obj/%.o: $(srcdir)/%.c $(GENH) $(IMPH)
 	$(CC_CMD)
@@ -153,7 +155,7 @@ obj/%.lo: $(srcdir)/%.s
 	$(AS_CMD)
 
 obj/%.lo: $(srcdir)/%.S
-	$(CC_CMD)
+	$(CCS_CMD)
 
 obj/%.lo: $(srcdir)/%.c $(GENH) $(IMPH)
 	$(CC_CMD)
-- 
2.51.0

[musl] [PATCH 3/3] aarch64: enable PAC and BTI instruction support in musl build

[Bill Roberts]

This change adds support for Pointer Authentication (PAC) and Branch
Target Identification (BTI) within musl’s own code on AArch64. These
features improve control-flow integrity and mitigate return-oriented
programming attacks by hardening indirect branches and return
instructions.

To integrate these instructions robustly across toolchains:

 - PAC and BTI instructions are inserted directly into the assembly,
   rather than being emitted via CFI directives. This approach is taken
   because it is far simpler to remove or rewrite instructions using AWK
   than to identify and manually annotate every location that requires
   them. New assembly code should therefore be written with PAC and BTI
   awareness in mind.

 - Since some older toolchains may not recognize PAC or BTI mnemonics,
   the post-processing step rewrites them into equivalent `hint`
   instructions. This allows musl to continue building successfully on
   systems without assembler support for these instructions, while still
   emitting the correct opcodes when using newer toolchains.

Together, these changes prepare musl for secure execution environments
with PAC and BTI enabled, while preserving backward compatibility.

References:
 - Arm Architecture Reference Manual for A-profile architecture (Arm ARM):
   https://developer.arm.com/documentation/ddi0487/latest
 - ELF for the Arm® 64-bit Architecture (AArch64) ABI supplement:
   https://github.com/ARM-software/abi-aa/blob/main/aaelf64/aaelf64.rst
 - Arm Community Blog – *Enabling Pointer Authentication and Branch Target Identification*:
   1. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part1
   2. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part2
   3. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part3

Signed-off-by: Bill Roberts <bill.roberts@arm.com>
---
 Makefile                         |  21 ++++--
 arch/aarch64/crt_arch.h          |   3 +
 configure                        |  13 +++-
 crt/aarch64/crti.s               |   2 +
 crt/aarch64/crtn.s               |   2 +
 src/fenv/aarch64/fenv.s          |   7 ++
 src/ldso/aarch64/tlsdesc.s       |   2 +
 src/process/aarch64/vfork.s      |   1 +
 src/setjmp/aarch64/longjmp.s     |   1 +
 src/setjmp/aarch64/setjmp.s      |   1 +
 src/signal/aarch64/restore.s     |   1 +
 src/signal/aarch64/sigsetjmp.s   |   2 +
 src/string/aarch64/memcpy.S      |   1 +
 src/string/aarch64/memset.S      |   1 +
 src/thread/aarch64/__unmapself.s |   1 +
 src/thread/aarch64/clone.s       |   1 +
 src/thread/aarch64/syscall_cp.s  |   1 +
 tools/pac-bti-aarch64.awk        | 122 +++++++++++++++++++++++++++++++
 18 files changed, 176 insertions(+), 7 deletions(-)
 create mode 100644 tools/pac-bti-aarch64.awk

diff --git a/Makefile b/Makefile
index 4e62e0b3..620636ec 100644
--- a/Makefile
+++ b/Makefile
@@ -132,14 +132,23 @@ $(CRT_OBJS): CFLAGS_ALL += -DCRT
 $(LOBJS) $(LDSO_OBJS): CFLAGS_ALL += -fPIC
 
 CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
+CCS_CMD = $(CC_CMD)
+AS_CMD = $(CC_CMD)
 
-# Choose invocation of assembler to be used
+AWK_OPTS :=
 ifeq ($(ADD_CFI),yes)
-	AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
-	CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
-else
-	AS_CMD = $(CC_CMD)
-	CCS_CMD = $(CC_CMD)
+AWK_OPTS += -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk
+endif
+
+ifeq ($(ARCH),aarch64)
+AWK_OPTS += -f $(srcdir)/tools/pac-bti-aarch64.awk
+AWK_OPTS += -vaarch64_pac=$(AARCH64_PAC) -vaarch64_bti=$(AARCH64_BTI)
+endif
+
+# Choose invocation of assembler to be used
+ifneq ($(AWK_OPTS),)
+    AS_CMD = LC_ALL=C awk $(AWK_OPTS) $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
+    CCS_CMD = LC_ALL=C awk $(AWK_OPTS) $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
 endif
 
 obj/%.o: $(srcdir)/%.s
diff --git a/arch/aarch64/crt_arch.h b/arch/aarch64/crt_arch.h
index b64fb3dd..9384dcba 100644
--- a/arch/aarch64/crt_arch.h
+++ b/arch/aarch64/crt_arch.h
@@ -3,6 +3,9 @@ __asm__(
 ".global " START "\n"
 ".type " START ",%function\n"
 START ":\n"
+#if defined(__ARM_FEATURE_BTI_DEFAULT)
+"	hint 34\n" /* bti c */
+#endif
 "	mov x29, #0\n"
 "	mov x30, #0\n"
 "	mov x0, sp\n"
diff --git a/configure b/configure
index 3809f24c..7033f8cd 100755
--- a/configure
+++ b/configure
@@ -111,7 +111,10 @@ return 1
 fi
 }
 
-
+get_macro_value () {
+printf "#ifdef $1\n$1_VALUE=$1\n#else\n$1_VALUE=$2\n#endif\n" \
+| ${CC} ${CFLAGS} -E - | grep "$1_VALUE" | cut -d'=' -f2-
+}
 
 # Beginning of actual script
 
@@ -672,6 +675,12 @@ fi
 if test "$ARCH" = "aarch64" ; then
 trycppif __AARCH64EB__ "$t" && SUBARCH=${SUBARCH}_be
 CFLAGS_AUTO="${CFLAGS_AUTO} -DARCH_SUPPORTS_DL_ADD_PROTECTIONS"
+printf "Checking if bti is enabled..."
+aarch64_bti=$(get_macro_value __ARM_FEATURE_BTI_DEFAULT 0)
+printf " $aarch64_bti\n"
+printf "Checking if pac is enabled..."
+aarch64_pac=$(get_macro_value __ARM_FEATURE_PAC_DEFAULT 0)
+printf " $aarch64_pac\n"
 fi
 
 if test "$ARCH" = "loongarch64" ; then
@@ -831,6 +840,8 @@ ALL_TOOLS = $tools
 TOOL_LIBS = $tool_libs
 ADD_CFI = $ADD_CFI
 MALLOC_DIR = $malloc_dir
+AARCH64_BTI = $aarch64_bti
+AARCH64_PAC = $aarch64_pac
 EOF
 test "x$static" = xno && echo "STATIC_LIBS ="
 test "x$shared" = xno && echo "SHARED_LIBS ="
diff --git a/crt/aarch64/crti.s b/crt/aarch64/crti.s
index 3776fa64..9b309f41 100644
--- a/crt/aarch64/crti.s
+++ b/crt/aarch64/crti.s
@@ -3,6 +3,7 @@
 .type _init,%function
 .align 2
 _init:
+	paciasp
 	stp x29,x30,[sp,-16]!
 	mov x29,sp
 
@@ -11,5 +12,6 @@ _init:
 .type _fini,%function
 .align 2
 _fini:
+	paciasp
 	stp x29,x30,[sp,-16]!
 	mov x29,sp
diff --git a/crt/aarch64/crtn.s b/crt/aarch64/crtn.s
index 73cab692..4da1882a 100644
--- a/crt/aarch64/crtn.s
+++ b/crt/aarch64/crtn.s
@@ -1,7 +1,9 @@
 .section .init
 	ldp x29,x30,[sp],#16
+	autiasp
 	ret
 
 .section .fini
 	ldp x29,x30,[sp],#16
+	autiasp
 	ret
diff --git a/src/fenv/aarch64/fenv.s b/src/fenv/aarch64/fenv.s
index 8f3ec965..c9649f8b 100644
--- a/src/fenv/aarch64/fenv.s
+++ b/src/fenv/aarch64/fenv.s
@@ -1,6 +1,7 @@
 .global fegetround
 .type fegetround,%function
 fegetround:
+	bti c
 	mrs x0, fpcr
 	and w0, w0, #0xc00000
 	ret
@@ -9,6 +10,7 @@ fegetround:
 .hidden __fesetround
 .type __fesetround,%function
 __fesetround:
+	bti c
 	mrs x1, fpcr
 	bic w1, w1, #0xc00000
 	orr w1, w1, w0
@@ -19,6 +21,7 @@ __fesetround:
 .global fetestexcept
 .type fetestexcept,%function
 fetestexcept:
+	bti c
 	and w0, w0, #0x1f
 	mrs x1, fpsr
 	and w0, w0, w1
@@ -27,6 +30,7 @@ fetestexcept:
 .global feclearexcept
 .type feclearexcept,%function
 feclearexcept:
+	bti c
 	and w0, w0, #0x1f
 	mrs x1, fpsr
 	bic w1, w1, w0
@@ -37,6 +41,7 @@ feclearexcept:
 .global feraiseexcept
 .type feraiseexcept,%function
 feraiseexcept:
+	bti c
 	and w0, w0, #0x1f
 	mrs x1, fpsr
 	orr w1, w1, w0
@@ -47,6 +52,7 @@ feraiseexcept:
 .global fegetenv
 .type fegetenv,%function
 fegetenv:
+	bti c
 	mrs x1, fpcr
 	mrs x2, fpsr
 	stp w1, w2, [x0]
@@ -57,6 +63,7 @@ fegetenv:
 .global fesetenv
 .type fesetenv,%function
 fesetenv:
+	bti c
 	mov x1, #0
 	mov x2, #0
 	cmn x0, #1
diff --git a/src/ldso/aarch64/tlsdesc.s b/src/ldso/aarch64/tlsdesc.s
index c6c685b3..d68ff4a9 100644
--- a/src/ldso/aarch64/tlsdesc.s
+++ b/src/ldso/aarch64/tlsdesc.s
@@ -6,6 +6,7 @@
 .hidden __tlsdesc_static
 .type __tlsdesc_static,@function
 __tlsdesc_static:
+	bti c
 	ldr x0,[x0,#8]
 	ret
 
@@ -19,6 +20,7 @@ __tlsdesc_static:
 .hidden __tlsdesc_dynamic
 .type __tlsdesc_dynamic,@function
 __tlsdesc_dynamic:
+	bti c
 	stp x1,x2,[sp,#-16]!
 	mrs x1,tpidr_el0      // tp
 	ldr x0,[x0,#8]        // p
diff --git a/src/process/aarch64/vfork.s b/src/process/aarch64/vfork.s
index 429bec8c..332aa4ae 100644
--- a/src/process/aarch64/vfork.s
+++ b/src/process/aarch64/vfork.s
@@ -1,6 +1,7 @@
 .global vfork
 .type vfork,%function
 vfork:
+	bti c
 	mov x8, 220    // SYS_clone
 	mov x0, 0x4111 // SIGCHLD | CLONE_VM | CLONE_VFORK
 	mov x1, 0
diff --git a/src/setjmp/aarch64/longjmp.s b/src/setjmp/aarch64/longjmp.s
index 0af9c50e..990642a4 100644
--- a/src/setjmp/aarch64/longjmp.s
+++ b/src/setjmp/aarch64/longjmp.s
@@ -4,6 +4,7 @@
 .type longjmp,%function
 _longjmp:
 longjmp:
+	bti c
 	// IHI0055B_aapcs64.pdf 5.1.1, 5.1.2 callee saved registers
 	ldp x19, x20, [x0,#0]
 	ldp x21, x22, [x0,#16]
diff --git a/src/setjmp/aarch64/setjmp.s b/src/setjmp/aarch64/setjmp.s
index f49288aa..3e94371b 100644
--- a/src/setjmp/aarch64/setjmp.s
+++ b/src/setjmp/aarch64/setjmp.s
@@ -7,6 +7,7 @@
 __setjmp:
 _setjmp:
 setjmp:
+	bti c
 	// IHI0055B_aapcs64.pdf 5.1.1, 5.1.2 callee saved registers
 	stp x19, x20, [x0,#0]
 	stp x21, x22, [x0,#16]
diff --git a/src/signal/aarch64/restore.s b/src/signal/aarch64/restore.s
index d4e5fcf1..341a1766 100644
--- a/src/signal/aarch64/restore.s
+++ b/src/signal/aarch64/restore.s
@@ -6,5 +6,6 @@ __restore:
 .hidden __restore_rt
 .type __restore_rt,%function
 __restore_rt:
+	bti c
 	mov x8,#139 // SYS_rt_sigreturn
 	svc 0
diff --git a/src/signal/aarch64/sigsetjmp.s b/src/signal/aarch64/sigsetjmp.s
index 75910c43..2b5d017f 100644
--- a/src/signal/aarch64/sigsetjmp.s
+++ b/src/signal/aarch64/sigsetjmp.s
@@ -4,6 +4,7 @@
 .type __sigsetjmp,%function
 sigsetjmp:
 __sigsetjmp:
+	bti c
 	cbz x1,setjmp
 
 	str x30,[x0,#176]
@@ -11,6 +12,7 @@ __sigsetjmp:
 	mov x19,x0
 
 	bl setjmp
+	bti j
 
 	mov w1,w0
 	mov x0,x19
diff --git a/src/string/aarch64/memcpy.S b/src/string/aarch64/memcpy.S
index 48bb8a8d..5959afb4 100644
--- a/src/string/aarch64/memcpy.S
+++ b/src/string/aarch64/memcpy.S
@@ -53,6 +53,7 @@
 .global memcpy
 .type memcpy,%function
 memcpy:
+	bti c
 	add     srcend, src, count
 	add     dstend, dstin, count
 	cmp     count, 128
diff --git a/src/string/aarch64/memset.S b/src/string/aarch64/memset.S
index f0d29b7f..37fdbab0 100644
--- a/src/string/aarch64/memset.S
+++ b/src/string/aarch64/memset.S
@@ -23,6 +23,7 @@
 .type memset,%function
 memset:
 
+	bti c
 	dup     v0.16B, valw
 	add     dstend, dstin, count
 
diff --git a/src/thread/aarch64/__unmapself.s b/src/thread/aarch64/__unmapself.s
index 2c5d254f..f9987538 100644
--- a/src/thread/aarch64/__unmapself.s
+++ b/src/thread/aarch64/__unmapself.s
@@ -1,6 +1,7 @@
 .global __unmapself
 .type   __unmapself,%function
 __unmapself:
+	bti c
 	mov x8,#215 // SYS_munmap
 	svc 0
 	mov x8,#93 // SYS_exit
diff --git a/src/thread/aarch64/clone.s b/src/thread/aarch64/clone.s
index aff8155b..e900a92a 100644
--- a/src/thread/aarch64/clone.s
+++ b/src/thread/aarch64/clone.s
@@ -8,6 +8,7 @@
 .hidden __clone
 .type   __clone,%function
 __clone:
+	bti c
 	// align stack and save func,arg
 	and x1,x1,#-16
 	stp x0,x3,[x1,#-16]!
diff --git a/src/thread/aarch64/syscall_cp.s b/src/thread/aarch64/syscall_cp.s
index 41db68af..e6baeef5 100644
--- a/src/thread/aarch64/syscall_cp.s
+++ b/src/thread/aarch64/syscall_cp.s
@@ -16,6 +16,7 @@
 .type __syscall_cp_asm,%function
 __syscall_cp_asm:
 __cp_begin:
+	bti c
 	ldr w0,[x0]
 	cbnz w0,__cp_cancel
 	mov x8,x1
diff --git a/tools/pac-bti-aarch64.awk b/tools/pac-bti-aarch64.awk
new file mode 100644
index 00000000..7b0222f1
--- /dev/null
+++ b/tools/pac-bti-aarch64.awk
@@ -0,0 +1,122 @@
+#!/usr/bin/env awk
+#
+# This script post processes aarch64 assembly to modify PAC and BTI instructions.
+# The aarch64 code is annotates as if PAC with the A key and BTI are enabled, and
+# then stripped or modified based on build time detection of compiler flags. Rather,
+# than attempt to insert the instructions, its easier to remove/modify after they are
+# added. This keeps the awk script and post processing much simpler. Note that we also
+# post process to use the hint instructions, as these are backwards compatible with
+# older binutils. Also to note, is that *these* PAC and BTI instructions, since they
+# are in the hint space also NOP on unsupoprted hardware. So there is no real penalty
+# to run a PAC/BTI aware binary on older hardware except for the cost of the NOP.
+#
+# Variables:
+# - aarch64_pac=[0|1|2] - Set this to 0 to disable pac, 1 to use the a key, and 2 to use the b key
+# - aarch64_bti=[0|1] - set this to 0 to disable bti, or 1 to enable bti
+
+# Details on PAC and BTI can be found in the manuals:
+#     - https://developer.arm.com/documentation/ddi0487/latest
+#     - https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst
+#
+# However, the TL;DR is the 3 part blog post that explores the relevant
+# parts for software:
+#  - https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-on-aarch64
+
+BEGIN {
+  # Validate aarch64_pac
+  if (aarch64_pac !~ /^(0|1|2)$/) {
+    print "Error: invalid value for aarch64_pac (" aarch64_pac "). Must be one of: 0, 1, 2." > "/dev/stderr"
+    exit 1
+  }
+
+  # Validate aarch64_bti
+  if (aarch64_bti !~ /^(0|1)$/) {
+    print "Error: invalid value for aarch64_bti (" aarch64_bti "). Must be one of: 0, 1." > "/dev/stderr"
+    exit 1
+  }
+}
+
+# Body
+# Behavior based on this table
+# | case | aarch64_bti | aarch64_pac | action |
+# | ---- | ----------- | ----------- | ------ |
+# |    1 |           0 |           0 | strip all paciasp, autiasp and bti c or j instructions |
+# |    2 |           0 |           a | strip all bti c or j instructions, rewrite pac using hints |
+# |    3 |           0 |           b | change paciasp to pacibsp and autiasp to autibsp instructions, using hint, and strip all bti c instructions|
+# |    4 |           1 |           0 | change all paciasp to bti c instructions, using hints and strip all autiasp instructions |
+# |    5 |           1 |           a | rewrite to hints |
+# |    6 |           1 |           b | change paciasp to pacibsp and autiasp to autibsp instructions |
+{
+# Declare some variables to keep the hint instruction mapping in one spot
+  PACIASP = "hint 25"
+  AUTIASP = "hint 29"
+  PACIBSP = "hint 27"
+  AUTIBSP = "hint 31"
+  BTI_C = "hint 34"
+  BTI_J = "hint 36"
+
+  # case 1 - strip all
+  if (aarch64_bti == 0 && aarch64_pac == 0 &&
+      /(paciasp|autiasp|bti[[:space:]]+[cj])/) {
+      next
+  # case 2 - strip bti c
+  } else if (aarch64_bti == 0 && aarch64_pac == 1) {
+    if (/bti[[:space:]]+[cj]/) {
+      next
+    }
+    gsub(/paciasp/, PACIASP)
+    gsub(/autiasp/, AUTIASP)
+  # case 3 - swap for b key and strip bti c
+  } else if (aarch64_bti == 0 && aarch64_pac == 2) {
+    if (/bti[[:space:]]+[cj]/) {
+      next
+    } else {
+      gsub(/paciasp/, PACIBSP)
+      gsub(/autiasp/, AUTIBSP)
+    }
+  # case 4 - remove autiasp and swap paciasp for bti c and rewrite bti j
+  } else if (aarch64_bti == 1 && aarch64_pac == 0) {
+    if (/autiasp/) {
+      next
+    } else {
+      gsub(/paciasp/, BTI_C)
+      gsub(/bti c/, BTI_C)
+      gsub(/bti j/, BTI_J)
+    }
+  # case 5 - rewrite all to hints
+  } else if (aarch64_bti == 1 && aarch64_pac == 1) {
+    gsub(/paciasp/, PACIASP)
+    gsub(/autiasp/, AUTIASP)
+    gsub(/bti c/, BTI_C)
+    gsub(/bti j/, BTI_J)
+  # case 6 - swap for b key
+  } else if (aarch64_bti == 1 && aarch64_pac == 2) {
+      gsub(/paciasp/, PACIBSP)
+      gsub(/autiasp/, AUTIBSP)
+      gsub(/bti c/, BTI_C)
+      gsub(/bti j/, BTI_J)
+  }
+
+  print
+}
+
+END {
+  # Add the GNU Notes section indicating what the binary supports.
+  GNU_PROPERTY_AARCH64_BTI = aarch64_bti
+  GNU_PROPERTY_AARCH64_POINTER_AUTH = (aarch64_pac != 0 ? 2 : 0)
+
+  if (aarch64_bti != 0 || aarch64_pac != 0) {
+    print "\n\n" \
+      ".pushsection .note.gnu.property, \"a\"; /* Start a new allocatable section */\n" \
+      ".balign 8; /* align it on a byte boundry */\n" \
+      ".long 4; /* size of \"GNU\0\" */\n" \
+      ".long 0x10; /* size of descriptor */\n" \
+      ".long 0x5; /* NT_GNU_PROPERTY_TYPE_0 */\n" \
+      ".asciz \"GNU\";\n" \
+      ".long 0xc0000000; /* GNU_PROPERTY_AARCH64_FEATURE_1_AND */\n" \
+      ".long 4; /* Four bytes of data */\n" \
+      ".long ("GNU_PROPERTY_AARCH64_BTI"|"GNU_PROPERTY_AARCH64_POINTER_AUTH"); /* BTI or PAC is enabled */\n" \
+      ".long 0; /* padding for 8 byte alignment */\n" \
+      ".popsection; /* end the section */"
+  }
+}
-- 
2.51.0

Re: [musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Thorsten Glaser]

On Thu, 13 Nov 2025, Bill Roberts wrote:

>--- a/ldso/dynlink.c
>+++ b/ldso/dynlink.c
>@@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
> 	off_t off_start;
> 	Ehdr *eh;
> 	Phdr *ph, *ph0;
>-	unsigned prot;
>+	unsigned prot, arch_prot, arch_mask;

You might want to zero-initialise these, as you changed
the = to |= below. (Unless there’s already an explicit
=0 in the part that’s not shown, I didn’t check.)

> 	unsigned char *map=MAP_FAILED, *base;
> 	size_t dyn=0;
> 	size_t tls_image=0;

bye,
//mirabilos
-- 
15:41⎜<Lo-lan-do:#fusionforge> Somebody write a testsuite for helloworld :-)

Re: [musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Bill Roberts]

On 11/13/25 5:09 PM, Thorsten Glaser wrote:
> On Thu, 13 Nov 2025, Bill Roberts wrote:
> 
>> --- a/ldso/dynlink.c
>> +++ b/ldso/dynlink.c
>> @@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
>> 	off_t off_start;
>> 	Ehdr *eh;
>> 	Phdr *ph, *ph0;
>> -	unsigned prot;
>> +	unsigned prot, arch_prot, arch_mask;
> 
> You might want to zero-initialise these, as you changed
> the = to |= below. (Unless there’s already an explicit
> =0 in the part that’s not shown, I didn’t check.)


They all should have explicit initializers before the |= like the
below snippet:

```C
prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
     ((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
     ((ph->p_flags&PF_X) ? PROT_EXEC : 0));
prot |= prot&arch_mask ? arch_prot : 0;
```

I poured over these uses to ensure there is an explicit initialization, 
if I missed one, please call that out.

For initialization, since prot is used in the loop, would be to 
initialize it to 0 in the start of the loop body. I left that out
of my patch since it's unrelated for now.

> 
>> 	unsigned char *map=MAP_FAILED, *base;
>> 	size_t dyn=0;
>> 	size_t tls_image=0;
> 
> bye,
> //mirabilos

Re: [musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Bill Roberts]

On 11/14/25 11:37 AM, Szabolcs Nagy wrote:
> * Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:26 -0600]:
> 
>> This change adds support for setting PROT_BTI on executable mappings
>> when the corresponding ELF object includes the GNU_PROPERTY_AARCH64_FEATURE_1_BTI
>> note. This ensures correct Branch Target Identification (BTI) enforcement
>> as defined in the Arm Architecture Reference Manual (Arm ARM) and the
>> ELF for the Arm 64-bit Architecture (AArch64) ABI supplement.
>>
>> When loading shared objects or executables, the dynamic loader now checks
>> for BTI-related GNU notes and applies the appropriate protection flags
>> via `mprotect` and `mmap` during mapping. This aligns musl’s behavior with
>> other modern loaders, improving security and correctness on systems with BTI
>> enabled hardware.
>>
>> References:
>>   - Arm Architecture Reference Manual for A-profile architecture (Arm ARM):
>>     https://developer.arm.com/documentation/ddi0487/latest
>>   - ELF for the Arm® 64-bit Architecture (AArch64) ABI supplement:
>>     https://github.com/ARM-software/abi-aa/blob/main/aaelf64/aaelf64.rst
>>   - Arm Community Blog – *Enabling Pointer Authentication and Branch Target Identification*:
>>     1. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part1
>>     2. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part2
>>     3. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part3
>>
>> Signed-off-by: Bill Roberts <bill.roberts@arm.com>
>> ---
> ...
>> +++ b/configure
>> @@ -671,6 +671,7 @@ fi
>>   
>>   if test "$ARCH" = "aarch64" ; then
>>   trycppif __AARCH64EB__ "$t" && SUBARCH=${SUBARCH}_be
>> +CFLAGS_AUTO="${CFLAGS_AUTO} -DARCH_SUPPORTS_DL_ADD_PROTECTIONS"
> 
> don't pass this around everywhere, add it to reloc.h
> 
>> +++ b/ldso/dynlink.c
>> @@ -682,6 +682,21 @@ static void unmap_library(struct dso *dso)
>>   	}
>>   }
>>   
>> +#ifdef ARCH_SUPPORTS_DL_ADD_PROTECTIONS
>> +#include <arch_ldso_hook.h>
>> +#else
>> +static inline unsigned dl_add_protections(const Elf64_Phdr *ph,
>> +	const Elf64_Ehdr *eh, int fd, unsigned *prot, unsigned *mask)
>> +{
>> +	(void)ph;
>> +	(void)eh;
>> +	(void)fd;
>> +	(void)prot;
>> +	(void)mask;
>> +	return 0;
>> +}
>> +#endif
>> +
> 
> you can place the hook+macro in arch/aarch64/reloc.h
> then no new file is needed.
> 
>>   static void *map_library(int fd, struct dso *dso)
>>   {
>>   	Ehdr buf[(896+sizeof(Ehdr))/sizeof(Ehdr)];
>> @@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
>>   	off_t off_start;
>>   	Ehdr *eh;
>>   	Phdr *ph, *ph0;
>> -	unsigned prot;
>> +	unsigned prot, arch_prot, arch_mask;
>>   	unsigned char *map=MAP_FAILED, *base;
>>   	size_t dyn=0;
>>   	size_t tls_image=0;
>> @@ -720,6 +735,16 @@ static void *map_library(int fd, struct dso *dso)
>>   	} else {
>>   		ph = ph0 = (void *)((char *)buf + eh->e_phoff);
>>   	}
>> +
>> +	/*
>> +	 * some architectures may have additional protection flags embedded in the
>> +	 * ELF section somewhere. Start with those and OR in the extras. Do this
>> +	 * before we need to map any sections.
>> +	 */
>> +	if(dl_add_protections(ph, eh, fd, &arch_prot, &arch_mask))
>> +		goto error;
>> +
>> +
>>   	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
>>   		if (ph->p_type == PT_DYNAMIC) {
>>   			dyn = ph->p_vaddr;
>> @@ -746,6 +771,7 @@ static void *map_library(int fd, struct dso *dso)
>>   			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>>   		}
>>   		if (ph->p_vaddr+ph->p_memsz > addr_max) {
>>   			addr_max = ph->p_vaddr+ph->p_memsz;
>> @@ -762,6 +788,7 @@ static void *map_library(int fd, struct dso *dso)
>>   			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>>   			map = mmap(0, ph->p_memsz + (ph->p_vaddr & PAGE_SIZE-1),
>>   				prot, MAP_PRIVATE,
>>   				fd, ph->p_offset & -PAGE_SIZE);
>> @@ -780,6 +807,7 @@ static void *map_library(int fd, struct dso *dso)
>>   				size_t pgbrk = brk + PAGE_SIZE-1 & -PAGE_SIZE;
>>   				size_t pgend = brk + ph->p_memsz - ph->p_filesz
>>   					+ PAGE_SIZE-1 & -PAGE_SIZE;
>> +				/* arch_prot has already been added */
>>   				if (pgend > pgbrk && mmap_fixed(map+pgbrk,
>>   					pgend-pgbrk, prot,
>>   					MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,
>> @@ -801,11 +829,16 @@ static void *map_library(int fd, struct dso *dso)
>>   	 * the length of the file. This is okay because we will not
>>   	 * use the invalid part; we just need to reserve the right
>>   	 * amount of virtual address space to map over later. */
>> -	map = DL_NOMMU_SUPPORT
>> -		? mmap((void *)addr_min, map_len, PROT_READ|PROT_WRITE|PROT_EXEC,
>> -			MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
>> -		: mmap((void *)addr_min, map_len, prot,
>> -			MAP_PRIVATE, fd, off_start);
>> +	if (map == DL_NOMMU_SUPPORT) {
>> +		prot = PROT_READ|PROT_WRITE|PROT_EXEC;
>> +		prot |= prot&arch_mask ? arch_prot : 0;
>> +		map = mmap((void *)addr_min, map_len, prot,
>> +				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
>> +	} else {
>> +		prot = prot & arch_mask ? prot | arch_prot : prot;
>> +		map = mmap((void *)addr_min, map_len, prot,
>> +				MAP_PRIVATE, fd, off_start);
>> +	}
> 
> i'd leave this code alone, prot should be
> set up already with arch_prot.
> the nommu case does not need bti support.
> 
>>   	if (map==MAP_FAILED) goto error;
>>   	dso->map = map;
>>   	dso->map_len = map_len;
>> @@ -835,6 +868,7 @@ static void *map_library(int fd, struct dso *dso)
>>   		prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   			((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   			((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +		prot |= prot&arch_mask ? arch_prot : 0;
>>   		/* Reuse the existing mapping for the lowest-address LOAD */
>>   		if ((ph->p_vaddr & -PAGE_SIZE) != addr_min || DL_NOMMU_SUPPORT)
>>   			if (mmap_fixed(base+this_min, this_max-this_min, prot, MAP_PRIVATE|MAP_FIXED, fd, off_start) == MAP_FAILED)
>> @@ -849,7 +883,9 @@ static void *map_library(int fd, struct dso *dso)
>>   	}
>>   	for (i=0; ((size_t *)(base+dyn))[i]; i+=2)
>>   		if (((size_t *)(base+dyn))[i]==DT_TEXTREL) {
>> -			if (mprotect(map, map_len, PROT_READ|PROT_WRITE|PROT_EXEC)
>> +			prot = PROT_READ|PROT_WRITE|PROT_EXEC;
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>> +			if (mprotect(map, map_len, prot)
>>   			    && errno != ENOSYS)
>>   				goto error;
> 
> i'd leave this alone too: DT_TEXTREL is a
> much bigger security hole than what bti
> is trying to cover here.
> 
> i don't think PROT_BTI generalizes to other targets
> 
>> +	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
>> +
>> +		/* skip non note sections */
>> +		if (ph->p_type != PT_NOTE)
>> +			continue;
>> +
>> +		buf = malloc(ph->p_filesz);
>> +		if (!buf)
>> +			return rc;
>> +
>> +		if (pread(fd, buf, ph->p_filesz, ph->p_offset) != ph->p_filesz)
>> +			goto error;
> ...
> 
> yeah gnu properties are a bit ugly..
> 
> the kernel handles the ld.so, the vdso and the
> static-linked exe cases.
> 
> however i don't think the kernel adds PROT_BTI
> to the main exe it loads, and map_library does
> not work for that (there is no fd to be mapped).
> please check what the kernel does (might behave
> differently now).
> 
> this caused trouble with systemd that seccomp
> blocked mprotect(PROT_EXEC|PROT_BTI), which was
> eventually solved on the kernel side
> https://lkml.org/lkml/2023/1/19/753
> 
> please test that exec pages have bti:
> 
> ./bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
> ld.so ./bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
> ./static-bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
> ./cat-with-dlopen /proc/self/smaps |grep 'VmFlags.*ex'
> 
> all lines should contain a 'bt' flag.

Thanks for that history, will check. AFAICT, it looks to
be the same, so you need to mprotect itself. There is a check
were if the interpreter in the ELF file is not itself to not
set PROT_BTI. Kernel code snippet below:

int arch_elf_adjust_prot(int prot, const struct arch_elf_state *state,
                          bool has_interp, bool is_interp)
{
         /*
          * For dynamically linked executables the interpreter is
          * responsible for setting PROT_BTI on everything except
          * itself.
          */
         if (is_interp != has_interp)
                 return prot;

         if (!(state->flags & ARM64_ELF_BTI))
                 return prot;

         if (prot & PROT_EXEC)
                 prot |= PROT_BTI;

         return prot;

}

Re: [musl] [PATCH 2/3] Makefile: support awk processing of .S files

[Bill Roberts]

On 11/14/25 10:23 AM, Szabolcs Nagy wrote:
> * Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:27 -0600]:
>> Don't skip .S files when post-processing with awk.
>>
>> Signed-off-by: Bill Roberts <bill.roberts@arm.com>
>> ---
>>   Makefile | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index a20bafaf..4e62e0b3 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -136,15 +136,17 @@ CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
>>   # Choose invocation of assembler to be used
>>   ifeq ($(ADD_CFI),yes)
>>   	AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
>> +	CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
>>   else
>>   	AS_CMD = $(CC_CMD)
>> +	CCS_CMD = $(CC_CMD)
>>   endif
>>   
>>   obj/%.o: $(srcdir)/%.s
>>   	$(AS_CMD)
>>   
>>   obj/%.o: $(srcdir)/%.S
>> -	$(CC_CMD)
>> +	$(CCS_CMD)
> 
> i think the x86 cfi awk deletes # and // comments
> which affects cpp directives in .S so you cant just
> use the awk on .S
> 
> i guess a hack would be to not use # comments in .s,
> another to only preprocess first then awk then asm.
> 

Personally, I would love to see all of this go to .S (capital)
files and use the C pre-processor for all of this and add the CFI
statements directly to the asm. However, it seems MUSL want's the 
assembly to be "pure" based on the comments and git logs I am
seeing. I am more of a fan of seeing everything in the assembly file 
then chasing down custom scripts that do processing, but I'll do it 
whatever way the community/maintainers want.

Re: [musl] [PATCH 3/3] aarch64: enable PAC and BTI instruction support in musl build

[Bill Roberts]

On 11/14/25 10:18 AM, Szabolcs Nagy wrote:
> * Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:28 -0600]:
>> This change adds support for Pointer Authentication (PAC) and Branch
>> Target Identification (BTI) within musl’s own code on AArch64. These
>> features improve control-flow integrity and mitigate return-oriented
>> programming attacks by hardening indirect branches and return
>> instructions.
>>
>> To integrate these instructions robustly across toolchains:
>>
>>   - PAC and BTI instructions are inserted directly into the assembly,
>>     rather than being emitted via CFI directives. This approach is taken
>>     because it is far simpler to remove or rewrite instructions using AWK
>>     than to identify and manually annotate every location that requires
>>     them. New assembly code should therefore be written with PAC and BTI
>>     awareness in mind.
> 
> i worked on the original gnu toolchain support, will
> add some notes.
> 
> note that pac requires dwarf cfi directives for correct
> unwinding (it clobbers the return address), which matters
> for non-leaf functions e.g. when unwinding from a signal
> handler (musl optionally supports async unwind), or when
> debugging (broken backtrace in gdb is bad user experience).
> this cfi is a pac-specific extension so some dwarf based
> unwinders may not support it and crash on it (a cfi
> interpreter cannot ignore unknown op codes, and it is
> hard to bump the dwarf abi to notice this at link time),
> so the pac extension can cause crashes even on cpus where
> the instruction is a nop so i think we should only add
> pac if the user asked for it.

Oh yeah, I was initially thinking C only, so no unwinders, but gdb and 
the C++ users will need those CFI directives to indicate the stack frame 
has a signed return address and which key was used.

> 
> it seems the only non-leaf asm that requires pac is legacy
> _init/_fini and aarch64 seems to define NO_LEGACY_INITFINI
> so we can probably remove the current _init/_fini code (?)

That'd be nice, I always like removing code.

Thanks for the review and insight Szabolcs, appreciate it.

> then all the pac complexity goes away from the asm. (with
> the understanding that if aarch64 needs non-leaf asm in the
> future, that will require additional work)
> 
>> --- a/crt/aarch64/crti.s
>> +++ b/crt/aarch64/crti.s
>> @@ -3,6 +3,7 @@
>>   .type _init,%function
>>   .align 2
>>   _init:
>> +	paciasp
>>   	stp x29,x30,[sp,-16]!
>>   	mov x29,sp
> ...
>> --- a/crt/aarch64/crtn.s
>> +++ b/crt/aarch64/crtn.s
>> @@ -1,7 +1,9 @@
>>   .section .init
>>   	ldp x29,x30,[sp],#16
>> +	autiasp
>>   	ret
> 
> crti.s could be
> 
> _init:
> 	bti c
> 	ret
> 
> just in case somebody calls _init (musl does not).
> 
>>
>>   - Since some older toolchains may not recognize PAC or BTI mnemonics,
>>     the post-processing step rewrites them into equivalent `hint`
>>     instructions. This allows musl to continue building successfully on
>>     systems without assembler support for these instructions, while still
>>     emitting the correct opcodes when using newer toolchains.
> 
> i think using an awk script is fine, but so far it did not
> modify the instructions, only added optional dwarf info.
> 
> another solution is to use .S and a header with BTI_C macros.
> or just add bti unconditionally (it is a nop, so only adds
> minor size and performance overhead, the code alignment
> changes in memcpy may be measurable and there was at least
> one particular supercomputer where hints were not as fast as
> normal nop, but i think for musl this is a valid choice)
> handling the gnu property note is uglier then though, awk
> is probably the best for that.

You can just include a header file and ifdef on ASM add the GNU note, 
this is how it's done for other projects. I'll repeat myself here, for 
readers, but as I said in Patch 2/3, I would prefer to use the CPP and 
move .S files.

> 
>>   .hidden __fesetround
>>   .type __fesetround,%function
>>   __fesetround:
>> +	bti c
>>   	mrs x1, fpcr
> 
> note: bti c is only required if the symbol may be called
> indirectly, so for hidden or local symbols a library may
> omit the bti c if it ensures there are no indirect calls
> to them. normally the toolchain may introduce indirect
> calls e.g. if a direct call goes too far in a huge binary
> the linker adds a stub that reaches the target with an
> indirect branch (via x16/x17 registers that are reserved
> for such tail calls and allowed to land on bti c). but it
> was decided that for bti enabled binaries a linker must
> not add such indirect branches to targets without bti c/j
> enabling a compiler to omit some bti c/j. i think we dont
> want to introduce constraints on the generic c code in
> musl so hidden symbols should keep the bti, local
> functions can avoid it but musl has no such case in asm.
> 
>> +++ b/arch/aarch64/crt_arch.h
>> @@ -3,6 +3,9 @@ __asm__(
>>   ".global " START "\n"
>>   ".type " START ",%function\n"
>>   START ":\n"
>> +#if defined(__ARM_FEATURE_BTI_DEFAULT)
>> +"	hint 34\n" /* bti c */
>> +#endif
> 
> i'd use
> #if __ARM_FEATURE_BTI_DEFAULT

Re: [musl] [PATCH 3/3] aarch64: enable PAC and BTI instruction support in musl build

[Szabolcs Nagy]

* Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:28 -0600]:
> This change adds support for Pointer Authentication (PAC) and Branch
> Target Identification (BTI) within musl’s own code on AArch64. These
> features improve control-flow integrity and mitigate return-oriented
> programming attacks by hardening indirect branches and return
> instructions.
> 
> To integrate these instructions robustly across toolchains:
> 
>  - PAC and BTI instructions are inserted directly into the assembly,
>    rather than being emitted via CFI directives. This approach is taken
>    because it is far simpler to remove or rewrite instructions using AWK
>    than to identify and manually annotate every location that requires
>    them. New assembly code should therefore be written with PAC and BTI
>    awareness in mind.

i worked on the original gnu toolchain support, will
add some notes.

note that pac requires dwarf cfi directives for correct
unwinding (it clobbers the return address), which matters
for non-leaf functions e.g. when unwinding from a signal
handler (musl optionally supports async unwind), or when
debugging (broken backtrace in gdb is bad user experience).
this cfi is a pac-specific extension so some dwarf based
unwinders may not support it and crash on it (a cfi
interpreter cannot ignore unknown op codes, and it is
hard to bump the dwarf abi to notice this at link time),
so the pac extension can cause crashes even on cpus where
the instruction is a nop so i think we should only add
pac if the user asked for it.

it seems the only non-leaf asm that requires pac is legacy
_init/_fini and aarch64 seems to define NO_LEGACY_INITFINI
so we can probably remove the current _init/_fini code (?)
then all the pac complexity goes away from the asm. (with
the understanding that if aarch64 needs non-leaf asm in the
future, that will require additional work)

> --- a/crt/aarch64/crti.s
> +++ b/crt/aarch64/crti.s
> @@ -3,6 +3,7 @@
>  .type _init,%function
>  .align 2
>  _init:
> +	paciasp
>  	stp x29,x30,[sp,-16]!
>  	mov x29,sp
...
> --- a/crt/aarch64/crtn.s
> +++ b/crt/aarch64/crtn.s
> @@ -1,7 +1,9 @@
>  .section .init
>  	ldp x29,x30,[sp],#16
> +	autiasp
>  	ret

crti.s could be

_init:
	bti c
	ret

just in case somebody calls _init (musl does not).

> 
>  - Since some older toolchains may not recognize PAC or BTI mnemonics,
>    the post-processing step rewrites them into equivalent `hint`
>    instructions. This allows musl to continue building successfully on
>    systems without assembler support for these instructions, while still
>    emitting the correct opcodes when using newer toolchains.

i think using an awk script is fine, but so far it did not
modify the instructions, only added optional dwarf info.

another solution is to use .S and a header with BTI_C macros.
or just add bti unconditionally (it is a nop, so only adds
minor size and performance overhead, the code alignment
changes in memcpy may be measurable and there was at least
one particular supercomputer where hints were not as fast as
normal nop, but i think for musl this is a valid choice)
handling the gnu property note is uglier then though, awk
is probably the best for that.

>  .hidden __fesetround
>  .type __fesetround,%function
>  __fesetround:
> +	bti c
>  	mrs x1, fpcr

note: bti c is only required if the symbol may be called
indirectly, so for hidden or local symbols a library may
omit the bti c if it ensures there are no indirect calls
to them. normally the toolchain may introduce indirect
calls e.g. if a direct call goes too far in a huge binary
the linker adds a stub that reaches the target with an
indirect branch (via x16/x17 registers that are reserved
for such tail calls and allowed to land on bti c). but it
was decided that for bti enabled binaries a linker must
not add such indirect branches to targets without bti c/j
enabling a compiler to omit some bti c/j. i think we dont
want to introduce constraints on the generic c code in
musl so hidden symbols should keep the bti, local
functions can avoid it but musl has no such case in asm.

> +++ b/arch/aarch64/crt_arch.h
> @@ -3,6 +3,9 @@ __asm__(
>  ".global " START "\n"
>  ".type " START ",%function\n"
>  START ":\n"
> +#if defined(__ARM_FEATURE_BTI_DEFAULT)
> +"	hint 34\n" /* bti c */
> +#endif

i'd use
#if __ARM_FEATURE_BTI_DEFAULT

Re: [musl] [PATCH 2/3] Makefile: support awk processing of .S files

[Szabolcs Nagy]

* Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:27 -0600]:
> Don't skip .S files when post-processing with awk.
> 
> Signed-off-by: Bill Roberts <bill.roberts@arm.com>
> ---
>  Makefile | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index a20bafaf..4e62e0b3 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -136,15 +136,17 @@ CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
>  # Choose invocation of assembler to be used
>  ifeq ($(ADD_CFI),yes)
>  	AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
> +	CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
>  else
>  	AS_CMD = $(CC_CMD)
> +	CCS_CMD = $(CC_CMD)
>  endif
>  
>  obj/%.o: $(srcdir)/%.s
>  	$(AS_CMD)
>  
>  obj/%.o: $(srcdir)/%.S
> -	$(CC_CMD)
> +	$(CCS_CMD)

i think the x86 cfi awk deletes # and // comments
which affects cpp directives in .S so you cant just
use the awk on .S

i guess a hack would be to not use # comments in .s,
another to only preprocess first then awk then asm.

Re: [musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Szabolcs Nagy]

* Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:26 -0600]:

> This change adds support for setting PROT_BTI on executable mappings
> when the corresponding ELF object includes the GNU_PROPERTY_AARCH64_FEATURE_1_BTI
> note. This ensures correct Branch Target Identification (BTI) enforcement
> as defined in the Arm Architecture Reference Manual (Arm ARM) and the
> ELF for the Arm 64-bit Architecture (AArch64) ABI supplement.
> 
> When loading shared objects or executables, the dynamic loader now checks
> for BTI-related GNU notes and applies the appropriate protection flags
> via `mprotect` and `mmap` during mapping. This aligns musl’s behavior with
> other modern loaders, improving security and correctness on systems with BTI
> enabled hardware.
> 
> References:
>  - Arm Architecture Reference Manual for A-profile architecture (Arm ARM):
>    https://developer.arm.com/documentation/ddi0487/latest
>  - ELF for the Arm® 64-bit Architecture (AArch64) ABI supplement:
>    https://github.com/ARM-software/abi-aa/blob/main/aaelf64/aaelf64.rst
>  - Arm Community Blog – *Enabling Pointer Authentication and Branch Target Identification*:
>    1. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part1
>    2. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part2
>    3. https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/enabling-pac-and-bti-part3
> 
> Signed-off-by: Bill Roberts <bill.roberts@arm.com>
> ---
...
> +++ b/configure
> @@ -671,6 +671,7 @@ fi
>  
>  if test "$ARCH" = "aarch64" ; then
>  trycppif __AARCH64EB__ "$t" && SUBARCH=${SUBARCH}_be
> +CFLAGS_AUTO="${CFLAGS_AUTO} -DARCH_SUPPORTS_DL_ADD_PROTECTIONS"

don't pass this around everywhere, add it to reloc.h

> +++ b/ldso/dynlink.c
> @@ -682,6 +682,21 @@ static void unmap_library(struct dso *dso)
>  	}
>  }
>  
> +#ifdef ARCH_SUPPORTS_DL_ADD_PROTECTIONS
> +#include <arch_ldso_hook.h>
> +#else
> +static inline unsigned dl_add_protections(const Elf64_Phdr *ph,
> +	const Elf64_Ehdr *eh, int fd, unsigned *prot, unsigned *mask)
> +{
> +	(void)ph;
> +	(void)eh;
> +	(void)fd;
> +	(void)prot;
> +	(void)mask;
> +	return 0;
> +}
> +#endif
> +

you can place the hook+macro in arch/aarch64/reloc.h
then no new file is needed.

>  static void *map_library(int fd, struct dso *dso)
>  {
>  	Ehdr buf[(896+sizeof(Ehdr))/sizeof(Ehdr)];
> @@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
>  	off_t off_start;
>  	Ehdr *eh;
>  	Phdr *ph, *ph0;
> -	unsigned prot;
> +	unsigned prot, arch_prot, arch_mask;
>  	unsigned char *map=MAP_FAILED, *base;
>  	size_t dyn=0;
>  	size_t tls_image=0;
> @@ -720,6 +735,16 @@ static void *map_library(int fd, struct dso *dso)
>  	} else {
>  		ph = ph0 = (void *)((char *)buf + eh->e_phoff);
>  	}
> +
> +	/*
> +	 * some architectures may have additional protection flags embedded in the
> +	 * ELF section somewhere. Start with those and OR in the extras. Do this
> +	 * before we need to map any sections.
> +	 */
> +	if(dl_add_protections(ph, eh, fd, &arch_prot, &arch_mask))
> +		goto error;
> +
> +
>  	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
>  		if (ph->p_type == PT_DYNAMIC) {
>  			dyn = ph->p_vaddr;
> @@ -746,6 +771,7 @@ static void *map_library(int fd, struct dso *dso)
>  			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>  				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>  				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
> +			prot |= prot&arch_mask ? arch_prot : 0;
>  		}
>  		if (ph->p_vaddr+ph->p_memsz > addr_max) {
>  			addr_max = ph->p_vaddr+ph->p_memsz;
> @@ -762,6 +788,7 @@ static void *map_library(int fd, struct dso *dso)
>  			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>  				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>  				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
> +			prot |= prot&arch_mask ? arch_prot : 0;
>  			map = mmap(0, ph->p_memsz + (ph->p_vaddr & PAGE_SIZE-1),
>  				prot, MAP_PRIVATE,
>  				fd, ph->p_offset & -PAGE_SIZE);
> @@ -780,6 +807,7 @@ static void *map_library(int fd, struct dso *dso)
>  				size_t pgbrk = brk + PAGE_SIZE-1 & -PAGE_SIZE;
>  				size_t pgend = brk + ph->p_memsz - ph->p_filesz
>  					+ PAGE_SIZE-1 & -PAGE_SIZE;
> +				/* arch_prot has already been added */
>  				if (pgend > pgbrk && mmap_fixed(map+pgbrk,
>  					pgend-pgbrk, prot,
>  					MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,
> @@ -801,11 +829,16 @@ static void *map_library(int fd, struct dso *dso)
>  	 * the length of the file. This is okay because we will not
>  	 * use the invalid part; we just need to reserve the right
>  	 * amount of virtual address space to map over later. */
> -	map = DL_NOMMU_SUPPORT
> -		? mmap((void *)addr_min, map_len, PROT_READ|PROT_WRITE|PROT_EXEC,
> -			MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
> -		: mmap((void *)addr_min, map_len, prot,
> -			MAP_PRIVATE, fd, off_start);
> +	if (map == DL_NOMMU_SUPPORT) {
> +		prot = PROT_READ|PROT_WRITE|PROT_EXEC;
> +		prot |= prot&arch_mask ? arch_prot : 0;
> +		map = mmap((void *)addr_min, map_len, prot,
> +				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
> +	} else {
> +		prot = prot & arch_mask ? prot | arch_prot : prot;
> +		map = mmap((void *)addr_min, map_len, prot,
> +				MAP_PRIVATE, fd, off_start);
> +	}

i'd leave this code alone, prot should be
set up already with arch_prot.
the nommu case does not need bti support.

>  	if (map==MAP_FAILED) goto error;
>  	dso->map = map;
>  	dso->map_len = map_len;
> @@ -835,6 +868,7 @@ static void *map_library(int fd, struct dso *dso)
>  		prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>  			((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>  			((ph->p_flags&PF_X) ? PROT_EXEC : 0));
> +		prot |= prot&arch_mask ? arch_prot : 0;
>  		/* Reuse the existing mapping for the lowest-address LOAD */
>  		if ((ph->p_vaddr & -PAGE_SIZE) != addr_min || DL_NOMMU_SUPPORT)
>  			if (mmap_fixed(base+this_min, this_max-this_min, prot, MAP_PRIVATE|MAP_FIXED, fd, off_start) == MAP_FAILED)
> @@ -849,7 +883,9 @@ static void *map_library(int fd, struct dso *dso)
>  	}
>  	for (i=0; ((size_t *)(base+dyn))[i]; i+=2)
>  		if (((size_t *)(base+dyn))[i]==DT_TEXTREL) {
> -			if (mprotect(map, map_len, PROT_READ|PROT_WRITE|PROT_EXEC)
> +			prot = PROT_READ|PROT_WRITE|PROT_EXEC;
> +			prot |= prot&arch_mask ? arch_prot : 0;
> +			if (mprotect(map, map_len, prot)
>  			    && errno != ENOSYS)
>  				goto error;

i'd leave this alone too: DT_TEXTREL is a
much bigger security hole than what bti
is trying to cover here.

i don't think PROT_BTI generalizes to other targets

> +	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
> +
> +		/* skip non note sections */
> +		if (ph->p_type != PT_NOTE)
> +			continue;
> +
> +		buf = malloc(ph->p_filesz);
> +		if (!buf)
> +			return rc;
> +
> +		if (pread(fd, buf, ph->p_filesz, ph->p_offset) != ph->p_filesz)
> +			goto error;
...

yeah gnu properties are a bit ugly..

the kernel handles the ld.so, the vdso and the
static-linked exe cases.

however i don't think the kernel adds PROT_BTI
to the main exe it loads, and map_library does
not work for that (there is no fd to be mapped).
please check what the kernel does (might behave
differently now).

this caused trouble with systemd that seccomp
blocked mprotect(PROT_EXEC|PROT_BTI), which was
eventually solved on the kernel side
https://lkml.org/lkml/2023/1/19/753

please test that exec pages have bti:

./bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
ld.so ./bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
./static-bti-cat /proc/self/smaps |grep 'VmFlags.*ex'
./cat-with-dlopen /proc/self/smaps |grep 'VmFlags.*ex'

all lines should contain a 'bt' flag.

Re: [musl] [PATCH 1/3] dso/aarch64: add PROT_BTI support for mem maps

[Bill Roberts]

<snip>

> 
> you can place the hook+macro in arch/aarch64/reloc.h
> then no new file is needed.
> 
>>   static void *map_library(int fd, struct dso *dso)
>>   {
>>   	Ehdr buf[(896+sizeof(Ehdr))/sizeof(Ehdr)];
>> @@ -693,7 +708,7 @@ static void *map_library(int fd, struct dso *dso)
>>   	off_t off_start;
>>   	Ehdr *eh;
>>   	Phdr *ph, *ph0;
>> -	unsigned prot;
>> +	unsigned prot, arch_prot, arch_mask;
>>   	unsigned char *map=MAP_FAILED, *base;
>>   	size_t dyn=0;
>>   	size_t tls_image=0;
>> @@ -720,6 +735,16 @@ static void *map_library(int fd, struct dso *dso)
>>   	} else {
>>   		ph = ph0 = (void *)((char *)buf + eh->e_phoff);
>>   	}
>> +
>> +	/*
>> +	 * some architectures may have additional protection flags embedded in the
>> +	 * ELF section somewhere. Start with those and OR in the extras. Do this
>> +	 * before we need to map any sections.
>> +	 */
>> +	if(dl_add_protections(ph, eh, fd, &arch_prot, &arch_mask))
>> +		goto error;
>> +
>> +
>>   	for (i=eh->e_phnum; i; i--, ph=(void *)((char *)ph+eh->e_phentsize)) {
>>   		if (ph->p_type == PT_DYNAMIC) {
>>   			dyn = ph->p_vaddr;
>> @@ -746,6 +771,7 @@ static void *map_library(int fd, struct dso *dso)
>>   			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>>   		}
>>   		if (ph->p_vaddr+ph->p_memsz > addr_max) {
>>   			addr_max = ph->p_vaddr+ph->p_memsz;
>> @@ -762,6 +788,7 @@ static void *map_library(int fd, struct dso *dso)
>>   			prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   				((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   				((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>>   			map = mmap(0, ph->p_memsz + (ph->p_vaddr & PAGE_SIZE-1),
>>   				prot, MAP_PRIVATE,
>>   				fd, ph->p_offset & -PAGE_SIZE);
>> @@ -780,6 +807,7 @@ static void *map_library(int fd, struct dso *dso)
>>   				size_t pgbrk = brk + PAGE_SIZE-1 & -PAGE_SIZE;
>>   				size_t pgend = brk + ph->p_memsz - ph->p_filesz
>>   					+ PAGE_SIZE-1 & -PAGE_SIZE;
>> +				/* arch_prot has already been added */
>>   				if (pgend > pgbrk && mmap_fixed(map+pgbrk,
>>   					pgend-pgbrk, prot,
>>   					MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS,
>> @@ -801,11 +829,16 @@ static void *map_library(int fd, struct dso *dso)
>>   	 * the length of the file. This is okay because we will not
>>   	 * use the invalid part; we just need to reserve the right
>>   	 * amount of virtual address space to map over later. */
>> -	map = DL_NOMMU_SUPPORT
>> -		? mmap((void *)addr_min, map_len, PROT_READ|PROT_WRITE|PROT_EXEC,
>> -			MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
>> -		: mmap((void *)addr_min, map_len, prot,
>> -			MAP_PRIVATE, fd, off_start);
>> +	if (map == DL_NOMMU_SUPPORT) {
>> +		prot = PROT_READ|PROT_WRITE|PROT_EXEC;
>> +		prot |= prot&arch_mask ? arch_prot : 0;
>> +		map = mmap((void *)addr_min, map_len, prot,
>> +				MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
>> +	} else {
>> +		prot = prot & arch_mask ? prot | arch_prot : prot;
>> +		map = mmap((void *)addr_min, map_len, prot,
>> +				MAP_PRIVATE, fd, off_start);
>> +	}
> 
> i'd leave this code alone, prot should be
> set up already with arch_prot.
> the nommu case does not need bti support.
> 
>>   	if (map==MAP_FAILED) goto error;
>>   	dso->map = map;
>>   	dso->map_len = map_len;
>> @@ -835,6 +868,7 @@ static void *map_library(int fd, struct dso *dso)
>>   		prot = (((ph->p_flags&PF_R) ? PROT_READ : 0) |
>>   			((ph->p_flags&PF_W) ? PROT_WRITE: 0) |
>>   			((ph->p_flags&PF_X) ? PROT_EXEC : 0));
>> +		prot |= prot&arch_mask ? arch_prot : 0;
>>   		/* Reuse the existing mapping for the lowest-address LOAD */
>>   		if ((ph->p_vaddr & -PAGE_SIZE) != addr_min || DL_NOMMU_SUPPORT)
>>   			if (mmap_fixed(base+this_min, this_max-this_min, prot, MAP_PRIVATE|MAP_FIXED, fd, off_start) == MAP_FAILED)
>> @@ -849,7 +883,9 @@ static void *map_library(int fd, struct dso *dso)
>>   	}
>>   	for (i=0; ((size_t *)(base+dyn))[i]; i+=2)
>>   		if (((size_t *)(base+dyn))[i]==DT_TEXTREL) {
>> -			if (mprotect(map, map_len, PROT_READ|PROT_WRITE|PROT_EXEC)
>> +			prot = PROT_READ|PROT_WRITE|PROT_EXEC;
>> +			prot |= prot&arch_mask ? arch_prot : 0;
>> +			if (mprotect(map, map_len, prot)
>>   			    && errno != ENOSYS)
>>   				goto error;
> 
> i'd leave this alone too: DT_TEXTREL is a
> much bigger security hole than what bti
> is trying to cover here.
> 
> i don't think PROT_BTI generalizes to other targets

Oh I forgot to elaborate on this. So, while for aarch64, we're adding 
PROT_BTI, other arches that wish to use the hook to propagate whatever 
flags they need, would potentially need this everywhere. With that said, 
I'm amenable to limiting the patch to just the aarch64 use case and if 
other arches need it, they can amend whatever spots they need.

Re: [musl] [PATCH 2/3] Makefile: support awk processing of .S files

[Rich Felker]

On Fri, Nov 14, 2025 at 12:38:03AM -0600, Bill Roberts wrote:
> 
> 
> On 11/14/25 10:23 AM, Szabolcs Nagy wrote:
> > * Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:27 -0600]:
> > > Don't skip .S files when post-processing with awk.
> > > 
> > > Signed-off-by: Bill Roberts <bill.roberts@arm.com>
> > > ---
> > >   Makefile | 6 ++++--
> > >   1 file changed, 4 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/Makefile b/Makefile
> > > index a20bafaf..4e62e0b3 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -136,15 +136,17 @@ CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
> > >   # Choose invocation of assembler to be used
> > >   ifeq ($(ADD_CFI),yes)
> > >   	AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
> > > +	CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
> > >   else
> > >   	AS_CMD = $(CC_CMD)
> > > +	CCS_CMD = $(CC_CMD)
> > >   endif
> > >   obj/%.o: $(srcdir)/%.s
> > >   	$(AS_CMD)
> > >   obj/%.o: $(srcdir)/%.S
> > > -	$(CC_CMD)
> > > +	$(CCS_CMD)
> > 
> > i think the x86 cfi awk deletes # and // comments
> > which affects cpp directives in .S so you cant just
> > use the awk on .S
> > 
> > i guess a hack would be to not use # comments in .s,
> > another to only preprocess first then awk then asm.
> > 
> 
> Personally, I would love to see all of this go to .S (capital)
> files and use the C pre-processor for all of this and add the CFI
> statements directly to the asm. However, it seems MUSL want's the assembly
> to be "pure" based on the comments and git logs I am
> seeing. I am more of a fan of seeing everything in the assembly file then
> chasing down custom scripts that do processing, but I'll do it whatever way
> the community/maintainers want.

I don't want more preprocessing scripts. The ones that are there are
there for optional debug functionality, not for anything that alters
behavior, and it would probably be nicer if we didn't need them.

At the same time, I don't want asm source files that require reading
include files to interpret what anything means. The long term goal is
to eliminate as many asm source files as possible, and it might make
sense to do this before doing anything that requires a lot of
redundant work on the existing ones. Really the only things that
should be asm source files rather than inline asm are things that need
to have an asm entry point/label in order to function -- things like
__syscal_cp_asm, __unmapself, etc.

Rich

Re: [musl] [PATCH 2/3] Makefile: support awk processing of .S files

[William Roberts]

On Mon, Nov 17, 2025 at 8:17 PM Rich Felker <dalias@libc.org> wrote:
>
> On Fri, Nov 14, 2025 at 12:38:03AM -0600, Bill Roberts wrote:
> >
> >
> > On 11/14/25 10:23 AM, Szabolcs Nagy wrote:
> > > * Bill Roberts <bill.roberts@arm.com> [2025-11-13 13:44:27 -0600]:
> > > > Don't skip .S files when post-processing with awk.
> > > >
> > > > Signed-off-by: Bill Roberts <bill.roberts@arm.com>
> > > > ---
> > > >   Makefile | 6 ++++--
> > > >   1 file changed, 4 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/Makefile b/Makefile
> > > > index a20bafaf..4e62e0b3 100644
> > > > --- a/Makefile
> > > > +++ b/Makefile
> > > > @@ -136,15 +136,17 @@ CC_CMD = $(CC) $(CFLAGS_ALL) -c -o $@ $<
> > > >   # Choose invocation of assembler to be used
> > > >   ifeq ($(ADD_CFI),yes)
> > > >           AS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler -c -o $@ -
> > > > + CCS_CMD = LC_ALL=C awk -f $(srcdir)/tools/add-cfi.common.awk -f $(srcdir)/tools/add-cfi.$(ARCH).awk $< | $(CC) $(CFLAGS_ALL) -x assembler-with-cpp -c -o $@ -
> > > >   else
> > > >           AS_CMD = $(CC_CMD)
> > > > + CCS_CMD = $(CC_CMD)
> > > >   endif
> > > >   obj/%.o: $(srcdir)/%.s
> > > >           $(AS_CMD)
> > > >   obj/%.o: $(srcdir)/%.S
> > > > - $(CC_CMD)
> > > > + $(CCS_CMD)
> > >
> > > i think the x86 cfi awk deletes # and // comments
> > > which affects cpp directives in .S so you cant just
> > > use the awk on .S
> > >
> > > i guess a hack would be to not use # comments in .s,
> > > another to only preprocess first then awk then asm.
> > >
> >
> > Personally, I would love to see all of this go to .S (capital)
> > files and use the C pre-processor for all of this and add the CFI
> > statements directly to the asm. However, it seems MUSL want's the assembly
> > to be "pure" based on the comments and git logs I am
> > seeing. I am more of a fan of seeing everything in the assembly file then
> > chasing down custom scripts that do processing, but I'll do it whatever way
> > the community/maintainers want.
>
> I don't want more preprocessing scripts. The ones that are there are
> there for optional debug functionality, not for anything that alters
> behavior, and it would probably be nicer if we didn't need them.
>
> At the same time, I don't want asm source files that require reading
> include files to interpret what anything means. The long term goal is
> to eliminate as many asm source files as possible, and it might make
> sense to do this before doing anything that requires a lot of
> redundant work on the existing ones. Really the only things that
> should be asm source files

Thanks for responding Rich, I was going to reach back out to see where you
stood on this. I for one am pretty fond of this conceptually even though
I am not a huge fan of incline assembly. I could probably start looking
at MUSL with this concept and float another RFC around that.

rather than inline asm are things that need
> to have an asm entry point/label in order to function -- things like
> __syscal_cp_asm, __unmapself, etc.

I don't think I fully appreciate what you're saying here, but it will probably
become clear as I look at the code. If I have any questions, I'll reach back
out.

Thanks Rich,
Bill