cortex-m support?

cortex-m support?

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 2691 bytes --]

I have access to a cortex-M board this week (and maybe next), and would
like to test out musl on it. It's a "SmartFusion 2" running one of the
https://github.com/emcraftsystems kernel trees (I forget which), and
buildroot with uclibc-ng. (In _theory_ that kernel includes fdpic
support, but the uclibc toolchain buildroot builds is binflt.)

There's an ellcc blog entry at http://ellcc.org/blog/?p=23234 that says
he added cortex-m support, which was commit
http://ellcc.org/viewvc/svn?view=revision&revision=4920 in their repo
(ancient patch attached, applies to a ~2 year old musl version).

Some of those changes don't seem to be needed anymore, because the arm
code has been moved to "unified syntax". But when I try to build with
buildroot's cortex-m3 toolchain, it clearly still needs some other changes:

src/setjmp/arm/longjmp.s: Assembler messages:
src/setjmp/arm/longjmp.s:10: Error: thumb conditional instruction should
be in IT block -- `moveq r0,#1'
src/setjmp/arm/longjmp.s:11: Error: SP not allowed in register list --
`ldmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}'

FYI gcc -v says that toolchain is configured with (among other things):

./configure --enable-static --target=arm-buildroot-uclinux-uclibcgnueabi
--disable-__cxa_atexit --with-gnu-ld --disable-libssp --disable-multilib
--disable-libquadmath --disable-libsanitizer --disable-tls
--disable-libmudflap --enable-threads --without-isl --without-cloog
--with-float=soft --disable-decimal-float --with-abi=aapcs-linux
--with-cpu=cortex-m3 --with-float=soft --with-mode=thumb
--enable-languages=c,c++ --disable-shared --disable-libgomp

(I can dig up the buildroot config if you want to try to reproduce it,
but I think I told Rich about it months ago.)

I have a todo item to try to get QEMU running a cortex-m3 Linux, but
unfortunately last time I checked QEMU's only actual cortex-m3 board
implementation was a toy ("netduino") with 128k of sram. It looks like
the vanilla kernel has 4 cortex-m defconfigs now. (Mostly cortex-m4, but
it seems like that's just funky DSP extensions the kernel presumably
doesn't use; cortex-m3 shipped in 2004 and is the common case nommu
baseline here).

Anyway, is there any interest in this? (Lemme rephrase that: the
_industry_ is screaming for this, is there any interest on the part of
musl?) I can try to hack something together from the old patch if so.
(Or somebody who knows the arm instruction set better can do more than
just "make the compile errors go away" :)

Rob

P.S. Anybody else boggle at the way wikipedia[citation needed]'s
cortex-m page doesn't actually include the phrase "mmu" or "memory
management unit"? No really: https://en.wikipedia.org/wiki/ARM_Cortex-M

[-- Attachment #2: musl-cortex-m.diff --]
[-- Type: text/x-diff, Size: 8723 bytes --]

------------------------------------------------------------------------
r4920 | rich | 2015-02-08 07:46:41 -0600 (Sun, 08 Feb 2015) | 1 line

Added support for the Cortex-M.
------------------------------------------------------------------------
Index: src/internal/arm/syscall.s
===================================================================
--- src/internal/arm/syscall.s	(revision 4919)
+++ src/internal/arm/syscall.s	(revision 4920)
@@ -11,5 +11,6 @@
 	svc 0
 	ldmfd sp!,{r4,r5,r6,r7}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
Index: src/ldso/arm/start.s
===================================================================
--- src/ldso/arm/start.s	(revision 4919)
+++ src/ldso/arm/start.s	(revision 4920)
@@ -14,5 +14,6 @@
 	mov r1,r0
 	mov r0,#0
 	tst r1,#1
+        it eq
 	moveq pc,r1
 	bx r1
Index: src/setjmp/arm/longjmp.s
===================================================================
--- src/setjmp/arm/longjmp.s	(revision 4919)
+++ src/setjmp/arm/longjmp.s	(revision 4920)
@@ -1,37 +0,0 @@
-.global _longjmp
-.global longjmp
-.type _longjmp,%function
-.type longjmp,%function
-_longjmp:
-longjmp:
-	mov ip,r0
-	movs r0,r1
-	moveq r0,#1
-	ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
-
-	adr r1,1f
-	ldr r2,1f
-	ldr r1,[r1,r2]
-
-	tst r1,#0x260
-	beq 3f
-	tst r1,#0x20
-	beq 2f
-	ldc p2, cr4, [ip], #48
-2:	tst r1,#0x40
-	beq 2f
-	.word 0xecbc8b10 /* vldmia ip!, {d8-d15} */
-2:	tst r1,#0x200
-	beq 3f
-	ldcl p1, cr10, [ip], #8
-	ldcl p1, cr11, [ip], #8
-	ldcl p1, cr12, [ip], #8
-	ldcl p1, cr13, [ip], #8
-	ldcl p1, cr14, [ip], #8
-	ldcl p1, cr15, [ip], #8
-3:	tst lr,#1
-	moveq pc,lr
-	bx lr
-
-.hidden __hwcap
-1:	.word __hwcap-1b
Index: src/setjmp/arm/setjmp.s
===================================================================
--- src/setjmp/arm/setjmp.s	(revision 4919)
+++ src/setjmp/arm/setjmp.s	(revision 4920)
@@ -1,39 +0,0 @@
-.global __setjmp
-.global _setjmp
-.global setjmp
-.type __setjmp,%function
-.type _setjmp,%function
-.type setjmp,%function
-__setjmp:
-_setjmp:
-setjmp:
-	mov ip,r0
-	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
-	mov r0,#0
-
-	adr r1,1f
-	ldr r2,1f
-	ldr r1,[r1,r2]
-
-	tst r1,#0x260
-	beq 3f
-	tst r1,#0x20
-	beq 2f
-	stc p2, cr4, [ip], #48
-2:	tst r1,#0x40
-	beq 2f
-	.word 0xecac8b10 /* vstmia ip!, {d8-d15} */
-2:	tst r1,#0x200
-	beq 3f
-	stcl p1, cr10, [ip], #8
-	stcl p1, cr11, [ip], #8
-	stcl p1, cr12, [ip], #8
-	stcl p1, cr13, [ip], #8
-	stcl p1, cr14, [ip], #8
-	stcl p1, cr15, [ip], #8
-3:	tst lr,#1
-	moveq pc,lr
-	bx lr
-
-.hidden __hwcap
-1:	.word __hwcap-1b
Index: src/setjmp/arm/setjmp.S
===================================================================
--- src/setjmp/arm/setjmp.S	(revision 0)
+++ src/setjmp/arm/setjmp.S	(revision 4920)
@@ -0,0 +1,45 @@
+.global __setjmp
+.global _setjmp
+.global setjmp
+.type __setjmp,%function
+.type _setjmp,%function
+.type setjmp,%function
+__setjmp:
+_setjmp:
+setjmp:
+	mov ip,r0
+#if defined(__thumb__)
+	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,lr}
+        str sp, [ip], #4
+#else
+	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
+#endif
+	mov r0,#0
+
+	adr r1,1f
+	ldr r2,1f
+	ldr r1,[r1,r2]
+
+	tst r1,#0x260
+	beq 3f
+	tst r1,#0x20
+	beq 2f
+	stc p2, cr4, [ip], #48
+2:	tst r1,#0x40
+	beq 2f
+	.word 0xecac8b10 /* vstmia ip!, {d8-d15} */
+2:	tst r1,#0x200
+	beq 3f
+	stcl p1, cr10, [ip], #8
+	stcl p1, cr11, [ip], #8
+	stcl p1, cr12, [ip], #8
+	stcl p1, cr13, [ip], #8
+	stcl p1, cr14, [ip], #8
+	stcl p1, cr15, [ip], #8
+3:	tst lr,#1
+        it eq
+	moveq pc,lr
+	bx lr
+
+.hidden __hwcap
+1:	.word __hwcap-1b
Index: src/setjmp/arm/longjmp.S
===================================================================
--- src/setjmp/arm/longjmp.S	(revision 0)
+++ src/setjmp/arm/longjmp.S	(revision 4920)
@@ -0,0 +1,44 @@
+.global _longjmp
+.global longjmp
+.type _longjmp,%function
+.type longjmp,%function
+_longjmp:
+longjmp:
+	mov ip,r0
+	movs r0,r1
+        it eq
+	moveq r0,#1
+#if defined(__thumb__)
+	ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,lr}
+        ldr sp, [ip], #4
+#else
+	ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
+#endif
+
+	adr r1,1f
+	ldr r2,1f
+	ldr r1,[r1,r2]
+
+	tst r1,#0x260
+	beq 3f
+	tst r1,#0x20
+	beq 2f
+	ldc p2, cr4, [ip], #48
+2:	tst r1,#0x40
+	beq 2f
+	.word 0xecbc8b10 /* vldmia ip!, {d8-d15} */
+2:	tst r1,#0x200
+	beq 3f
+	ldcl p1, cr10, [ip], #8
+	ldcl p1, cr11, [ip], #8
+	ldcl p1, cr12, [ip], #8
+	ldcl p1, cr13, [ip], #8
+	ldcl p1, cr14, [ip], #8
+	ldcl p1, cr15, [ip], #8
+3:	tst lr,#1
+        it eq
+	moveq pc,lr
+	bx lr
+
+.hidden __hwcap
+1:	.word __hwcap-1b
Index: src/thread/arm/syscall_cp.s
===================================================================
--- src/thread/arm/syscall_cp.s	(revision 4919)
+++ src/thread/arm/syscall_cp.s	(revision 4920)
@@ -7,6 +7,7 @@
 __cp_begin:
 	ldr r0,[r0]
 	cmp r0,#0
+        it ne
 	blne __cancel
 	mov r7,r1
 	mov r0,r2
@@ -17,5 +18,6 @@
 __cp_end:
 	ldmfd sp!,{r4,r5,r6,r7,lr}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
Index: src/thread/arm/clone.s
===================================================================
--- src/thread/arm/clone.s	(revision 4919)
+++ src/thread/arm/clone.s	(revision 4920)
@@ -16,6 +16,7 @@
 	beq 1f
 	ldmfd sp!,{r4,r5,r6,r7}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
 
Index: crt/arm/crtn.s
===================================================================
--- crt/arm/crtn.s	(revision 4919)
+++ crt/arm/crtn.s	(revision 4920)
@@ -1,6 +1,7 @@
 .section .init
 	pop {r0,lr}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
 
@@ -7,5 +8,6 @@
 .section .fini
 	pop {r0,lr}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
Index: Makefile
===================================================================
--- Makefile	(revision 4919)
+++ Makefile	(revision 4920)
@@ -106,6 +106,10 @@
 $(dir $(patsubst %/,%,$(dir $(1))))$(notdir $(1:.s=.o)): $(1)
 endef
 $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call mkasmdep,$(s))))
+define mkasmdepS
+$(dir $(patsubst %/,%,$(dir $(1))))$(notdir $(1:.S=.o)): $(1)
+endef
+$(foreach s,$(wildcard src/*/$(ARCH)*/*.S),$(eval $(call mkasmdepS,$(s))))
 
 %.o: $(ARCH)$(ASMSUBARCH)/%.sub
 	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $(dir $<)$(shell cat $<)
@@ -113,6 +117,9 @@
 %.o: $(ARCH)/%.s
 	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
 
+%.o: $(ARCH)/%.S
+	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
+
 %.o: %.c $(GENH) $(IMPH)
 	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
 
@@ -122,6 +129,9 @@
 %.lo: $(ARCH)/%.s
 	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<
 
+%.lo: $(ARCH)/%.S
+	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<
+
 %.lo: %.c $(GENH) $(IMPH)
 	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<
 
Index: arch/arm/src/arm/atomics.s
===================================================================
--- arch/arm/src/arm/atomics.s	(revision 4919)
+++ arch/arm/src/arm/atomics.s	(revision 4920)
@@ -6,12 +6,13 @@
 __a_barrier:
 	ldr ip,1f
 	ldr ip,[pc,ip]
-	add pc,pc,ip
+	add pc,ip
 1:	.word __a_barrier_ptr-1b
 .global __a_barrier_dummy
 .hidden __a_barrier_dummy
 __a_barrier_dummy:
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
 .global __a_barrier_oldkuser
@@ -25,6 +26,7 @@
 	mov pc,ip
 	pop {r0,r1,r2,r3,ip,lr}
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
 .global __a_barrier_v6
@@ -44,7 +46,7 @@
 __a_cas:
 	ldr ip,1f
 	ldr ip,[pc,ip]
-	add pc,pc,ip
+	add pc,ip
 1:	.word __a_cas_ptr-1b
 .global __a_cas_dummy
 .hidden __a_cas_dummy
@@ -52,8 +54,10 @@
 	mov r3,r0
 	ldr r0,[r2]
 	subs r0,r3,r0
+        it eq
 	streq r1,[r2]
 	tst lr,#1
+        it eq
 	moveq pc,lr
 	bx lr
 .global __a_cas_v6
@@ -64,6 +68,7 @@
 1:	.word 0xe1920f9f        /* ldrex r0,[r2] */
 	subs r0,r3,r0
 	.word 0x01820f91        /* strexeq r0,r1,[r2] */
+        it eq
 	teqeq r0,#1
 	beq 1b
 	mcr p15,0,r0,c7,c10,5
@@ -76,6 +81,7 @@
 1:	.word 0xe1920f9f        /* ldrex r0,[r2] */
 	subs r0,r3,r0
 	.word 0x01820f91        /* strexeq r0,r1,[r2] */
+        it eq
 	teqeq r0,#1
 	beq 1b
 	.word 0xf57ff05b        /* dmb ish */
@@ -91,7 +97,7 @@
 __a_gettp:
 	ldr r0,1f
 	ldr r0,[pc,r0]
-	add pc,pc,r0
+	add pc,r0
 1:	.word __a_gettp_ptr-1b
 .global __a_gettp_dummy
 .hidden __a_gettp_dummy
Index: arch/arm/crt_arch.h
===================================================================
--- arch/arm/crt_arch.h	(revision 4919)
+++ arch/arm/crt_arch.h	(revision 4920)
@@ -1,11 +1,17 @@
-__asm__("\
-.text \n\
-.global _start \n\
-.type _start,%function \n\
-_start: \n\
-	mov fp, #0 \n\
-	mov lr, #0 \n\
-	mov a1, sp \n\
-	bic sp, sp, #0xF \n\
-	bl __cstart \n\
-");
+__asm__(
+".text \n"
+".global _start \n"
+".type _start,%function \n"
+"_start: \n"
+"	mov fp, #0 \n"
+"	mov lr, #0 \n"
+"	mov a1, sp \n"
+#if defined(__thumb__)
+"       mov a2, sp \n"
+"       bic a2, #0xF \n"
+"       mov sp, a2 \n"
+#else
+"	bic sp, sp, #0xF \n"
+#endif
+"	bl __cstart \n"
+);

Re: cortex-m support?

[Szabolcs Nagy]

* Rob Landley <rob@landley.net> [2016-12-06 23:52:29 -0600]:
> Anyway, is there any interest in this? (Lemme rephrase that: the
> _industry_ is screaming for this, is there any interest on the part of
> musl?) I can try to hack something together from the old patch if so.
> (Or somebody who knows the arm instruction set better can do more than
> just "make the compile errors go away" :)

i think musl is interested in the asm fixes.

fdpic arm is interesting too but it would be better
if there was an abi spec (so there is no toolchain
compatibility issue).

i'm not sure if linux on cortex-m is really what the
industry wants.. i'd assume more cortex-m users would
want musl on baremetal or on custom rtos.

Re: cortex-m support?

[Szabolcs Nagy]

* Szabolcs Nagy <nsz@port70.net> [2016-12-07 16:29:37 +0100]:
> 
> fdpic arm is interesting too but it would be better
> if there was an abi spec (so there is no toolchain
> compatibility issue).
> 

ok it seems somebody wrote this:
https://github.com/mickael-guene/fdpic_doc/blob/master/abi.txt

it's just not upstreamed anywhere (and not specified by the vendor)

Re: cortex-m support?

[Rob Landley]

On 12/07/2016 09:35 AM, Szabolcs Nagy wrote:
> * Szabolcs Nagy <nsz@port70.net> [2016-12-07 16:29:37 +0100]:
>>
>> fdpic arm is interesting too but it would be better
>> if there was an abi spec (so there is no toolchain
>> compatibility issue).
>>
> 
> ok it seems somebody wrote this:
> https://github.com/mickael-guene/fdpic_doc/blob/master/abi.txt
> 
> it's just not upstreamed anywhere (and not specified by the vendor)

Ah, that's one of the three authors of this:

http://www.slideshare.net/linaroorg/sfo15406-arm-fdpic-toolset-kernel-libraries-for-cortexm-cortexr-mmuless-cores

Although emcraft.com (the company whose full-time business has been
"Linux on Cortex-M" for over 5 years) is in Moscow. Their fdpic support
patch in their kernel is:

https://github.com/EmcraftSystems/linux-emcraft/commit/673205bf926b

And it looks like it was a port of the work of those guys.

Not sure where you get the relevant gcc patch...

Rob

Re: cortex-m support?

[Rich Felker]

On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
> 
> On 12/07/2016 09:35 AM, Szabolcs Nagy wrote:
> > * Szabolcs Nagy <nsz@port70.net> [2016-12-07 16:29:37 +0100]:
> >>
> >> fdpic arm is interesting too but it would be better
> >> if there was an abi spec (so there is no toolchain
> >> compatibility issue).
> >>
> > 
> > ok it seems somebody wrote this:
> > https://github.com/mickael-guene/fdpic_doc/blob/master/abi.txt
> > 
> > it's just not upstreamed anywhere (and not specified by the vendor)
> 
> Ah, that's one of the three authors of this:
> 
> http://www.slideshare.net/linaroorg/sfo15406-arm-fdpic-toolset-kernel-libraries-for-cortexm-cortexr-mmuless-cores
> 
> Although emcraft.com (the company whose full-time business has been
> "Linux on Cortex-M" for over 5 years) is in Moscow. Their fdpic support
> patch in their kernel is:
> 
> https://github.com/EmcraftSystems/linux-emcraft/commit/673205bf926b
> 
> And it looks like it was a port of the work of those guys.
> 
> Not sure where you get the relevant gcc patch...

I have links to the repos somewhere; the problem is that they're
forked from a fairly old gcc version (although not nearly as bad as
sh-fdpic; I think it's 4.8 or so) and the authors probably don't have
copyright assignments taken care of. It would be really nice if
someone could coordinate working with the authors and companies
involved and getting all of that taken care of -- preferably someone
getting paid to do so, since it's not particularly fun work to be
doing as a volunteer.

Rich

Re: cortex-m support?

[Rob Landley]

On 12/07/2016 07:16 PM, Rich Felker wrote:
> On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
>> Not sure where you get the relevant gcc patch...
> 
> I have links to the repos somewhere; the problem is that they're
> forked from a fairly old gcc version (although not nearly as bad as
> sh-fdpic; I think it's 4.8 or so)

https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
updated 20 hours ago, although I'm not sure how much of master updating
is an automatic tracking branch and how much is the account owner
updating things.

> and the authors probably don't have
> copyright assignments taken care of.

Nobody does copyright assignments voluntarily. In the long run those
kill projects.

> It would be really nice if
> someone could coordinate working with the authors and companies
> involved and getting all of that taken care of -- preferably someone
> getting paid to do so, since it's not particularly fun work to be
> doing as a volunteer.

Github says https://github.com/mickael-guene was last active November
30th (commiting stuff to his cortex-m simulator fork), so he's still
around...

I'm tempted to try to shoehorn the gcc and binutils here into
musl-cross-make and see what happens. But I've already _got_ a cortex-m
targeted toolchain that... can't build musl. Because of the assembly
stuff. So I should probably poke at that first.

> Rich

Rob

Re: cortex-m support?

[Rich Felker]

On Thu, Dec 08, 2016 at 01:10:11PM -0600, Rob Landley wrote:
> On 12/07/2016 07:16 PM, Rich Felker wrote:
> > On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
> >> Not sure where you get the relevant gcc patch...
> > 
> > I have links to the repos somewhere; the problem is that they're
> > forked from a fairly old gcc version (although not nearly as bad as
> > sh-fdpic; I think it's 4.8 or so)
> 
> https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
> updated 20 hours ago, although I'm not sure how much of master updating
> is an automatic tracking branch and how much is the account owner
> updating things.

Excellent! That's new since I last communicated with him.

> > and the authors probably don't have
> > copyright assignments taken care of.
> 
> Nobody does copyright assignments voluntarily. In the long run those
> kill projects.

I don't want to start an unrelated GNU-politics thread, but I think
you're mistaken here. Any modification of a GNU package will die out
in the long run without copyright assignment, because nobody can
maintain stuff out-of-tree forever. If it's an individual they'll lose
interest; if it's a company their business interests will change or
they'll get bought out by someone who doesn't care. In my opinion this
is 100% the FSF's fault for creating such a broken system, but it's
the reality. So people who modify GNU stuff and who do want their work
to live on and be useful in the future really should be pushing to get
their employers' to authorize the assignments.

> > It would be really nice if
> > someone could coordinate working with the authors and companies
> > involved and getting all of that taken care of -- preferably someone
> > getting paid to do so, since it's not particularly fun work to be
> > doing as a volunteer.
> 
> Github says https://github.com/mickael-guene was last active November
> 30th (commiting stuff to his cortex-m simulator fork), so he's still
> around...
> 
> I'm tempted to try to shoehorn the gcc and binutils here into
> musl-cross-make and see what happens. But I've already _got_ a cortex-m
> targeted toolchain that... can't build musl. Because of the assembly
> stuff. So I should probably poke at that first.

This is really good news -- I think adding the patches should be easy.
You can probably just diff his tree against stock 5.2.0 and make it
one big combined patch. This makes getting stuff ready on the musl
side a lot more attractive to me, so I will try to get back to is
asap.

Rich

Re: cortex-m support?

[Rob Landley]

On 12/08/2016 03:01 PM, Rich Felker wrote:
> On Thu, Dec 08, 2016 at 01:10:11PM -0600, Rob Landley wrote:
>> On 12/07/2016 07:16 PM, Rich Felker wrote:
>>> On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
>>>> Not sure where you get the relevant gcc patch...
>>>
>>> I have links to the repos somewhere; the problem is that they're
>>> forked from a fairly old gcc version (although not nearly as bad as
>>> sh-fdpic; I think it's 4.8 or so)
>>
>> https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
>> updated 20 hours ago, although I'm not sure how much of master updating
>> is an automatic tracking branch and how much is the account owner
>> updating things.
> 
> Excellent! That's new since I last communicated with him.
> 
>>> and the authors probably don't have
>>> copyright assignments taken care of.
>>
>> Nobody does copyright assignments voluntarily. In the long run those
>> kill projects.
> 
> I don't want to start an unrelated GNU-politics thread, but...
>...
> In my opinion this is 100% the FSF's fault for creating such a broken
> system, but it's the reality.

More or less what I meant, yes.

Keep in mind gcc development -already_ died once (egcs), and the lesson
the llvm guys learned is "don't fork off the old dead base, even if it's
been dead for years it's poison and they'll steal it back if you get it
going again, instead start over from scratch with something they have no
claim to".

> So people who modify GNU stuff and who do want their work
> to live on and be useful in the future

already screwed up before the "and"; everything else is damage control.

> This is really good news -- I think adding the patches should be easy.
> You can probably just diff his tree against stock 5.2.0 and make it
> one big combined patch. This makes getting stuff ready on the musl
> side a lot more attractive to me, so I will try to get back to is
> asap.

FYI, the cortex-m toolchain I've been building (which builds a kernel
and userspace that runs on the board) is just buildroot's cortex-m
toolchain.

Grab current buildroot, "make defconfig" then "make menuconfig" and:

  target options -> target architecture -> arm little endian
  target options -> target architecture variant -> cortex-m3
  build options -> host dir (set to /opt/arm-linux)
  toolchain->enable c++ support

Then "sudo make" (so it can write to /opt/arm-linux).

The version of buildroot I had needed to do "ln -s arm-linux-ld.real
/opt/arm-linux/usr/bin/ld.real" because the binflt wrapper is broken and
evil, but maybe that's fixed upstream since? And the buildroot commit
history disables -Os on cortex-m due to some compiler bug (maybe fixed
now? Haven't hit it myself but I build with -O2 instead for this board
on general principles.)

Oh, and there's a separate "Enable elf2flt support" config thing in the
toolchain menu that you DON'T want to select or it'll wrap the toolchain
_twice_ and break, buildroot does it automatically for cortex-m. I think
it's even a magic wrapper doing the -Wl,-elf2flt stuff automatically. :)

> Rich
> .

Rob

Re: cortex-m support?

[Rob Landley]

On 12/08/2016 03:01 PM, Rich Felker wrote:
> On Thu, Dec 08, 2016 at 01:10:11PM -0600, Rob Landley wrote:
>> On 12/07/2016 07:16 PM, Rich Felker wrote:
>>> On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
>>>> Not sure where you get the relevant gcc patch...
>>>
>>> I have links to the repos somewhere; the problem is that they're
>>> forked from a fairly old gcc version (although not nearly as bad as
>>> sh-fdpic; I think it's 4.8 or so)
>>
>> https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
>> updated 20 hours ago, although I'm not sure how much of master updating
>> is an automatic tracking branch and how much is the account owner
>> updating things.
> 
> Excellent! That's new since I last communicated with him.

In theory, what you do is follow the instructions in:

  https://github.com/mickael-guene/fdpic_manifest

Which uses repo to download the github repositories, and then has a
build.sh. In practice, I'm not sure what branches that uses to build
said toolchain.

However, this would give you a uClibc-based cortex-m-fdpic toolchain you
could then swap musl into. It also has a lsit of kernel patches that
theoretically work against vanilla, but you don't have a board for that. :)

I did poke you over the summer about running cortex-m binflt binaries
under qemu application emulation (works fine). I dunno if it supports
the fdpic loader or not, building this toolchain to find out. If not,
it's got an emulator there too but I haven't poked at it yet...

Rob

Re: cortex-m support?

[Rich Felker]

On Mon, Dec 12, 2016 at 06:29:13PM -0600, Rob Landley wrote:
> 
> 
> On 12/08/2016 03:01 PM, Rich Felker wrote:
> > On Thu, Dec 08, 2016 at 01:10:11PM -0600, Rob Landley wrote:
> >> On 12/07/2016 07:16 PM, Rich Felker wrote:
> >>> On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
> >>>> Not sure where you get the relevant gcc patch...
> >>>
> >>> I have links to the repos somewhere; the problem is that they're
> >>> forked from a fairly old gcc version (although not nearly as bad as
> >>> sh-fdpic; I think it's 4.8 or so)
> >>
> >> https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
> >> updated 20 hours ago, although I'm not sure how much of master updating
> >> is an automatic tracking branch and how much is the account owner
> >> updating things.
> > 
> > Excellent! That's new since I last communicated with him.
> 
> In theory, what you do is follow the instructions in:
> 
>   https://github.com/mickael-guene/fdpic_manifest

OK, I'll take a look.

> Which uses repo to download the github repositories, and then has a
> build.sh. In practice, I'm not sure what branches that uses to build
> said toolchain.
> 
> However, this would give you a uClibc-based cortex-m-fdpic toolchain you
> could then swap musl into.

Just diffing the tree and dropping the patch in musl-cross-make is
going to be a much faster path to getting something reproducible and
testable.

> It also has a lsit of kernel patches that
> theoretically work against vanilla, but you don't have a board for that. :)

Can you get me (instructions for building) a kernel that boots on qemu
but has the fdpic-support patches for arm? The main patch hunk that'll
be needed is the part in signal handler invocation that uses the
function pointer as either a code address or a function descriptor.

BTW, upstreaming this patch without an old pending kernel change I
still need to submit will introduce a CVE-worthy vulnerability on all
ARM systems. :-)

> I did poke you over the summer about running cortex-m binflt binaries
> under qemu application emulation (works fine). I dunno if it supports
> the fdpic loader or not, building this toolchain to find out. If not,
> it's got an emulator there too but I haven't poked at it yet...

It can load fdpic as normal ELF just fine, but signal handlers will
crash if it doesn't treat the application-passed function pointer as a
function descriptor. Fixing this should just be a several-line patch
to qemu if nobody already did it.

Rich

Re: cortex-m support?

[Rich Felker]

On Thu, Dec 08, 2016 at 01:10:11PM -0600, Rob Landley wrote:
> On 12/07/2016 07:16 PM, Rich Felker wrote:
> > On Wed, Dec 07, 2016 at 06:55:56PM -0600, Rob Landley wrote:
> >> Not sure where you get the relevant gcc patch...
> > 
> > I have links to the repos somewhere; the problem is that they're
> > forked from a fairly old gcc version (although not nearly as bad as
> > sh-fdpic; I think it's 4.8 or so)
> 
> https://github.com/mickael-guene/gcc has a 5.2 branch and "master"
> updated 20 hours ago, although I'm not sure how much of master updating
> is an automatic tracking branch and how much is the account owner
> updating things.

Those don't seem to have the fdpic work; the fdpic work is in branch
gcc-linaro-4.7-2013.05-cec-fdpic. The 5.2.0 branch is nothing but the
entire 5.2.0 source tree as a single git commit.

This isn't a show-stopper, but it does mean this is all still based on
ancient gcc, and we need someone to add gcc 4.7 support to mcm (i.e.
backport the musl patches to 4.7), and to work out whether the fdpic
patches apply cleanly on 4.7 or depend on all sorts of other linaro
changes, before this can more forward much.

On the bright side, the musl-side changes for supporting cortex-m (not
fdpic, just normal elf binaries that should run on nommu) are mostly
done. I can probably get you a hello world program you can run as
init= on a nommu arm kernel to see if it actually works.

Rich

Re: cortex-m support?

[Rob Landley]

On 12/07/2016 09:29 AM, Szabolcs Nagy wrote:
> i'm not sure if linux on cortex-m is really what the
> industry wants..

http://electronicdesign.com/embedded/practical-advice-running-uclinux-cortex-m3m4
https://community.arm.com/thread/5364
http://events.linuxfoundation.org/sites/events/files/slides/optimize-uclinux.pdf
http://www.microsemi.com/document-portal/doc_download/130852-linux-cortex-m-user
http://stackoverflow.com/questions/6636830/how-do-i-run-linux-on-an-arm-cortex-m3-board
http://hackerboards.com/iot-dev-kit-runs-uclinux-on-a-microsemi-cortex-m3-fpga-soc/
http://www.cnx-software.com/2014/06/28/amptek-icon-is-an-arm-cortex-m3-board-for-iot-running-uclinux-crowdfunding/
http://electronics.stackexchange.com/questions/19234/linux-on-arm-cortex-m3-series
http://elinux.org/STM32
http://www.cnx-software.com/2011/12/30/linux-for-cortex-m3-m4-microcontrollers/
https://angel.co/projects/137810-design-of-an-arm-cortex-m4-stm32f4-based-embedded-linux-system?src=user_profile
https://falstaff.agner.ch/2015/05/17/u-boot-command-to-boot-vybrids-cortex-m4-core/
https://developer.mbed.org/users/noritsuna/notebook/linux-on-cortexm3armv7m/

It's a mystery,

Rob

Re: cortex-m support?

[Rich Felker]

On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
> Added support for the Cortex-M.
> ------------------------------------------------------------------------
> Index: src/internal/arm/syscall.s
> ===================================================================
> --- src/internal/arm/syscall.s	(revision 4919)
> +++ src/internal/arm/syscall.s	(revision 4920)
> @@ -11,5 +11,6 @@
>  	svc 0
>  	ldmfd sp!,{r4,r5,r6,r7}
>  	tst lr,#1
> +        it eq
>  	moveq pc,lr
>  	bx lr

There's a gas option -Wa,-mimplicit-it=always that will make these
kind of changes unnecessary. I think it's preferable to just have
musl's configure always add that option when targeting arm if it's
accepted by the toolchain. Otherwise new code might get added without
checking that it builds as thumb.

> Index: src/setjmp/arm/setjmp.S
> ===================================================================
> --- src/setjmp/arm/setjmp.S	(revision 0)
> +++ src/setjmp/arm/setjmp.S	(revision 4920)
> @@ -0,0 +1,45 @@
> +.global __setjmp
> +.global _setjmp
> +.global setjmp
> +.type __setjmp,%function
> +.type _setjmp,%function
> +.type setjmp,%function
> +__setjmp:
> +_setjmp:
> +setjmp:
> +	mov ip,r0
> +#if defined(__thumb__)
> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,lr}
> +        str sp, [ip], #4
> +#else
> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
> +#endif

My leaning is to just always use the version that works on thumb, as I
don't think the performance difference will be measurable. I think you
did change the order of the storage in jmp_buf; I'm okay with that but
we might want to see if anyone objects because of wacky stuff
expecting a particular order. (Note: I don't think the order should be
treated as public but we probably want to know what if anything is
poking at it so we know if there are third-party things that need
fixing.)

> Index: src/thread/arm/clone.s
> ===================================================================
> --- src/thread/arm/clone.s	(revision 4919)
> +++ src/thread/arm/clone.s	(revision 4920)
> @@ -16,6 +16,7 @@
>  	beq 1f
>  	ldmfd sp!,{r4,r5,r6,r7}
>  	tst lr,#1
> +        it eq
>  	moveq pc,lr
>  	bx lr

Clone will also need a fix something like what I did for sh in commit
234c58467c3709bafdd3ffa6ac73655e1dfd9ddb to be compatible with fdpic,
but that's separate from thumb support.

> Index: Makefile
> ===================================================================
> --- Makefile	(revision 4919)
> +++ Makefile	(revision 4920)
> @@ -106,6 +106,10 @@
>  $(dir $(patsubst %/,%,$(dir $(1))))$(notdir $(1:.s=.o)): $(1)
>  endef
>  $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call mkasmdep,$(s))))
> +define mkasmdepS
> +$(dir $(patsubst %/,%,$(dir $(1))))$(notdir $(1:.S=.o)): $(1)
> +endef
> +$(foreach s,$(wildcard src/*/$(ARCH)*/*.S),$(eval $(call mkasmdepS,$(s))))
>  
>  %.o: $(ARCH)$(ASMSUBARCH)/%.sub
>  	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $(dir $<)$(shell cat $<)
> @@ -113,6 +117,9 @@
>  %.o: $(ARCH)/%.s
>  	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
>  
> +%.o: $(ARCH)/%.S
> +	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
> +
>  %.o: %.c $(GENH) $(IMPH)
>  	$(CC) $(CFLAGS_ALL_STATIC) -c -o $@ $<
>  
> @@ -122,6 +129,9 @@
>  %.lo: $(ARCH)/%.s
>  	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<
>  
> +%.lo: $(ARCH)/%.S
> +	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<
> +
>  %.lo: %.c $(GENH) $(IMPH)
>  	$(CC) $(CFLAGS_ALL_SHARED) -c -o $@ $<

This looks like it's against a really old version of musl. The modern
build system with out-of-tree support already accepts .S files.

> Index: arch/arm/src/arm/atomics.s
> ===================================================================
> --- arch/arm/src/arm/atomics.s	(revision 4919)
> +++ arch/arm/src/arm/atomics.s	(revision 4920)
> @@ -6,12 +6,13 @@
>  __a_barrier:
>  	ldr ip,1f
>  	ldr ip,[pc,ip]
> -	add pc,pc,ip
> +	add pc,ip

I don't think this actually works; the arithmetic on pc does different
things in thumb mode. This file just needs to be scrapped and
redesigned since it's not going to be compatible with fdpic anyway (it
relies on PC-relative accesses to global data).

> Index: arch/arm/crt_arch.h
> ===================================================================
> --- arch/arm/crt_arch.h	(revision 4919)
> +++ arch/arm/crt_arch.h	(revision 4920)
> @@ -1,11 +1,17 @@
> -__asm__("\
> -.text \n\
> -.global _start \n\
> -.type _start,%function \n\
> -_start: \n\
> -	mov fp, #0 \n\
> -	mov lr, #0 \n\
> -	mov a1, sp \n\
> -	bic sp, sp, #0xF \n\
> -	bl __cstart \n\
> -");
> +__asm__(
> +".text \n"
> +".global _start \n"
> +".type _start,%function \n"
> +"_start: \n"
> +"	mov fp, #0 \n"
> +"	mov lr, #0 \n"
> +"	mov a1, sp \n"
> +#if defined(__thumb__)
> +"       mov a2, sp \n"
> +"       bic a2, #0xF \n"
> +"       mov sp, a2 \n"
> +#else
> +"	bic sp, sp, #0xF \n"
> +#endif
> +"	bl __cstart \n"
> +);

I think I had a simpler version of this tucked away somewhere; I'll
check and see.

Thanks for working on this and reviving interest in the topic.

Rich

Re: cortex-m support?

[Rich Felker]

[-- Attachment #1: Type: text/plain, Size: 1448 bytes --]

On Thu, Dec 08, 2016 at 04:11:16PM -0500, Rich Felker wrote:
> On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
> > Added support for the Cortex-M.
> > ------------------------------------------------------------------------
> > Index: src/internal/arm/syscall.s
> > ===================================================================
> > --- src/internal/arm/syscall.s	(revision 4919)
> > +++ src/internal/arm/syscall.s	(revision 4920)
> > @@ -11,5 +11,6 @@
> >  	svc 0
> >  	ldmfd sp!,{r4,r5,r6,r7}
> >  	tst lr,#1
> > +        it eq
> >  	moveq pc,lr
> >  	bx lr
> 
> There's a gas option -Wa,-mimplicit-it=always that will make these
> kind of changes unnecessary. I think it's preferable to just have
> musl's configure always add that option when targeting arm if it's
> accepted by the toolchain. Otherwise new code might get added without
> checking that it builds as thumb.

I dug up the changes I made to get it to build (not run; atomics.s is
broken) as thumb-only; see attached. Note that the atomics.s part of
the diff actively breaks the code so that it won't work even as arm
code.

I'll see if I can come up with a good solution for the atomics. One
thing I do need to know is how the thread pointer is supposed to be
read on cortex-m, since afaik the coprocessor register normally used
does not exist. Does the kernel trap and emulate the coprocessor
register, or is there some other mechanism that must be used?

Rich

[-- Attachment #2: thumb.diff --]
[-- Type: text/plain, Size: 1953 bytes --]

diff --git a/src/setjmp/arm/longjmp.s b/src/setjmp/arm/longjmp.s
index e28d8f3..6191ab2 100644
--- a/src/setjmp/arm/longjmp.s
+++ b/src/setjmp/arm/longjmp.s
@@ -8,7 +8,9 @@ longjmp:
 	mov ip,r0
 	movs r0,r1
 	moveq r0,#1
-	ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
+	ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp}
+	ldmia ip!, {r2,lr}
+	mov sp,r2
 
 	adr r1,1f
 	ldr r2,1f
diff --git a/src/setjmp/arm/setjmp.s b/src/setjmp/arm/setjmp.s
index 8779163..c6fe1bb 100644
--- a/src/setjmp/arm/setjmp.s
+++ b/src/setjmp/arm/setjmp.s
@@ -9,7 +9,9 @@ __setjmp:
 _setjmp:
 setjmp:
 	mov ip,r0
-	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
+	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp}
+	mov r2,sp
+	stmia ip!,{r2,lr}
 	mov r0,#0
 
 	adr r1,1f
diff --git a/src/string/arm/memcpy.c b/src/string/arm/memcpy.c
index 041614f..f703c9b 100644
--- a/src/string/arm/memcpy.c
+++ b/src/string/arm/memcpy.c
@@ -1,3 +1,3 @@
-#if __ARMEB__
+#if __ARMEB__ || __thumb__
 #include "../memcpy.c"
 #endif
diff --git a/src/string/arm/memcpy_le.S b/src/string/arm/memcpy_le.S
index 4db4844..9cfbcb2 100644
--- a/src/string/arm/memcpy_le.S
+++ b/src/string/arm/memcpy_le.S
@@ -1,4 +1,4 @@
-#ifndef __ARMEB__
+#if !__ARMEB__ && !__thumb__
 
 /*
  * Copyright (C) 2008 The Android Open Source Project
diff --git a/src/thread/arm/atomics.s b/src/thread/arm/atomics.s
index 673fc03..380f714 100644
--- a/src/thread/arm/atomics.s
+++ b/src/thread/arm/atomics.s
@@ -6,7 +6,7 @@
 .type __a_barrier,%function
 __a_barrier:
 	ldr ip,1f
-	ldr ip,[pc,ip]
+	//ldr ip,[pc,ip]
 	add pc,pc,ip
 1:	.word __a_barrier_ptr-1b
 .global __a_barrier_dummy
@@ -40,7 +40,7 @@ __a_barrier_v7:
 .type __a_cas,%function
 __a_cas:
 	ldr ip,1f
-	ldr ip,[pc,ip]
+	//ldr ip,[pc,ip]
 	add pc,pc,ip
 1:	.word __a_cas_ptr-1b
 .global __a_cas_dummy
@@ -85,7 +85,7 @@ __aeabi_read_tp:
 .type __a_gettp,%function
 __a_gettp:
 	ldr r0,1f
-	ldr r0,[pc,r0]
+	//ldr r0,[pc,r0]
 	add pc,pc,r0
 1:	.word __a_gettp_ptr-1b
 .global __a_gettp_dummy

Re: [musl] cortex-m support?

[Rob Landley]

On 12/08/2016 03:11 PM, Rich Felker wrote:
> On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
>> Index: src/setjmp/arm/setjmp.S
>> ===================================================================
>> --- src/setjmp/arm/setjmp.S	(revision 0)
>> +++ src/setjmp/arm/setjmp.S	(revision 4920)
>> @@ -0,0 +1,45 @@
>> +.global __setjmp
>> +.global _setjmp
>> +.global setjmp
>> +.type __setjmp,%function
>> +.type _setjmp,%function
>> +.type setjmp,%function
>> +__setjmp:
>> +_setjmp:
>> +setjmp:
>> +	mov ip,r0
>> +#if defined(__thumb__)
>> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,lr}
>> +        str sp, [ip], #4
>> +#else
>> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
>> +#endif
> 
> My leaning is to just always use the version that works on thumb,

Presumaby there's a longjmp.S bit too because:

        ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}

Make sure you're aware of erratum 752419:

  https://sourceware.org/ml/binutils/2011-05/msg00087.html

On a board with a lot of serial traffic, it takes about 3 minutes for
uClibc (yes, including the current -ng) to crash (by loading the link
register into the stack pointer and stomping code before the setjmp).

The emcraft guys fixed this in their cortex-m uClibc fork way back when
(http://www.emcraft.com/som/building-uclibc), and of _course_ never
bothered to send it upstream. (Or put it on github with their kernel and
uboot.) But the source is in their distro tarball, and their patch to
uClibc's __longjmp.S is basically:

--- uclibc-1.0.17/libc/sysdeps/linux/arm/__longjmp.S
+++ emcraft/libc/sysdeps/linux/arm/__longjmp.S
@@ -56,9 +60,15 @@
 #if defined(__thumb2__)
 	/* Thumb-2 does not allow loading sp with ldm.  */
 	ldmia	ip!,	{v1-v6, sl, fp}
+#if 0
 	ldr	sp, [ip], #4
 	ldr	lr, [ip], #4
 #else
+	ldr	sp, [ip, #0]
+	ldr	lr, [ip, #4]
+	add	ip, #8
+#endif
+#else
 	ldmia	ip!,	{v1-v6, sl, fp, sp, lr}
 #endif


(Dunno why the "add ip, #8" because it's a scratch register and this is
the last use in the function, but maybe they just made it do exactly
what the #else case does?)

cc-ing buildroot because this is still broken in their november release.

Rob

Re: [musl] cortex-m support?

[Waldemar Brodkorb]

Hi,
Rob Landley wrote,

> On 12/08/2016 03:11 PM, Rich Felker wrote:
> > On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
> >> Index: src/setjmp/arm/setjmp.S
> >> ===================================================================
> >> --- src/setjmp/arm/setjmp.S	(revision 0)
> >> +++ src/setjmp/arm/setjmp.S	(revision 4920)
> >> @@ -0,0 +1,45 @@
> >> +.global __setjmp
> >> +.global _setjmp
> >> +.global setjmp
> >> +.type __setjmp,%function
> >> +.type _setjmp,%function
> >> +.type setjmp,%function
> >> +__setjmp:
> >> +_setjmp:
> >> +setjmp:
> >> +	mov ip,r0
> >> +#if defined(__thumb__)
> >> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,lr}
> >> +        str sp, [ip], #4
> >> +#else
> >> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
> >> +#endif
> > 
> > My leaning is to just always use the version that works on thumb,
> 
> Presumaby there's a longjmp.S bit too because:
> 
>         ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
> 
> Make sure you're aware of erratum 752419:
> 
>   https://sourceware.org/ml/binutils/2011-05/msg00087.html
> 
> On a board with a lot of serial traffic, it takes about 3 minutes for
> uClibc (yes, including the current -ng) to crash (by loading the link
> register into the stack pointer and stomping code before the setjmp).
> 
> The emcraft guys fixed this in their cortex-m uClibc fork way back when
> (http://www.emcraft.com/som/building-uclibc), and of _course_ never
> bothered to send it upstream. (Or put it on github with their kernel and
> uboot.) But the source is in their distro tarball, and their patch to
> uClibc's __longjmp.S is basically:
> 
> --- uclibc-1.0.17/libc/sysdeps/linux/arm/__longjmp.S
> +++ emcraft/libc/sysdeps/linux/arm/__longjmp.S
> @@ -56,9 +60,15 @@
>  #if defined(__thumb2__)
>  	/* Thumb-2 does not allow loading sp with ldm.  */
>  	ldmia	ip!,	{v1-v6, sl, fp}
> +#if 0
>  	ldr	sp, [ip], #4
>  	ldr	lr, [ip], #4
>  #else
> +	ldr	sp, [ip, #0]
> +	ldr	lr, [ip, #4]
> +	add	ip, #8
> +#endif
> +#else
>  	ldmia	ip!,	{v1-v6, sl, fp, sp, lr}
>  #endif
> 
> 
> (Dunno why the "add ip, #8" because it's a scratch register and this is
> the last use in the function, but maybe they just made it do exactly
> what the #else case does?)
> 
> cc-ing buildroot because this is still broken in their november release.

I am wondering why you don't cc me or any uclibc related list? 

You still believe uClibc projects should die?

It is really cool and very nice that the communication between Rich
and me is always fruitful. I always report any bugs I find while
testing musl on new or old platforms (mostly via IRC channel) and I
always take care that bugfixes for musl targets end up in buildroot.
(mipsr6 support, binutils microblaze problems, ..)

I really would like to see a similar open communication between the 
nommu community, you and me. 

Just ignoring uClibc-ng does not will make it die.

As a good starting point, a nice bug report and/or test application
which allows me to reproduce the problem would be really
appreciated.

best regards
 Waldemar

Re: [Buildroot] [musl] cortex-m support?

[Rob Landley]

Sorry for the delay responding, lots of deadlines and travel recently...

(Ok, for a definition of "recently" that goes back a decade:
http://lists.busybox.net/pipermail/busybox/2005-June/048743.html)

On 12/15/2016 12:51 PM, Waldemar Brodkorb wrote:
> Hi,
> Rob Landley wrote,
> 
>> On 12/08/2016 03:11 PM, Rich Felker wrote:
>>> On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
>> Make sure you're aware of erratum 752419:
>>
>>   https://sourceware.org/ml/binutils/2011-05/msg00087.html
...
>> cc-ing buildroot because this is still broken in their november release.
> 
> I am wondering why you don't cc me or any uclibc related list? 

I cc'd the buildroot list, which only has uClibc-based cortex-m support
at the moment. Why do you suppose I did that?

Did you want me to send it to the uclibc.org mailing list which hasn't
had a single post this month except your announcement of your fork's
release? The list where nobody's noticed the chrome browser can't access
https://lists.uclibc.org (archives, subscription page, etc) for weeks
now? And yes, I publicized that fact when I noticed it:

  https://twitter.com/landley/status/806202364822597632

Several people replied to that tweet, if nobody bothered to poke
uclibc.org I'm clearly not the only one who thinks the project is dead.

Your fork clearly hasn't fixed any of the structural issues uClibc
developed over the years. We are discussing a patch to cortex-m that I
found because code was crashing, and when gdb was deployed we found out
that code in libpthread/libthreads/pthread.c was getting corrupted, and
we worked out that the corruption ended right after the sigsetjmp() call
in __pthread_timedsuspend_new() and that all the data before it looked
like stack contents, stack grows down on every linux target except
pa-risc so that's actually where the corruption started... and we went
from there.

This was debugging through the _old_ legacy pthreads implementation,
which is the only option on cortex-m because even the one buildroot has
today never got NPTL working there. Musl has nptl for every target it
supports (including a nommu one), but the uClibc developers have spent
TEN YEARS trying to make that work since the first out-of-tree NTPL
version that sjhill promised to release after his OLS presentation, then
after his customer paid him enough, then...

If you don't understand WHY this is such a bad thing, then you clearly
don't know the history of the project. You don't now what the problems
that led to all the multi-year gaps between releases WERE.

Has your fork solved the locales problem?

  http://lists.busybox.net/pipermail/uclibc/2015-June/049000.html

Has your fork solved the nptl issue?

  http://lists.busybox.net/pipermail/uclibc/2008-September/020151.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020169.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020171.html
  http://lists.busybox.net/pipermail/uclibc/2008-October/041201.html

Did you fix the fact that huge swaths of uClibc (especially headers) are
just snapshots of old versions of glibc stuff advertising things uClibc
doesn't actually implement and pretending to be glibc?

  http://lists.busybox.net/pipermail/uclibc/2006-March/014811.html

Do you have a sane "make defconfig" that lets people build uClibc
without learning what over a hundred individual config options do and
making decisions about whether or not they need each one? This issue
doesn't even come _up_ with musl, it fundamentally avoids most of the
structural problems that strangled uClibc development, by design.

> You still believe uClibc projects should die?

No, I believe uClibc _already_ died. I believe this because I was there.

I did my turn as Sisyphus on that project. I pushed the boulder uphill
over an over for over a decade. And yes, I mean a full decade, starting
in 2003:

  http://lists.busybox.net/pipermail/uclibc/2003-August/027643.html

I used to prod the maintainer to get releases out by sending him
birthday cakes when the existing release was a year old:

  http://lists.busybox.net/pipermail/uclibc/2005-January/010877.html
  http://lists.busybox.net/pipermail/uclibc/2006-March/014921.html
  http://lists.busybox.net/pipermail/uclibc/2006-March/014923.html
  http://lists.busybox.net/pipermail/uclibc/2006-December/037750.html
  http://lists.busybox.net/pipermail/uclibc/2006-December/037766.html

I had to fight hard for the very IDEA of cutting releases:

  http://lists.busybox.net/pipermail/uclibc/2006-March/014722.html
  http://lists.busybox.net/pipermail/uclibc/2006-March/014881.html
  http://lists.busybox.net/pipermail/uclibc/2006-March/014885.html

Here's a changelog I spent 3 days researching after a particularly long
development cycle (because users had expressed reluctance to move to new
versions when so _much_ had changed at once that the couldn't debug
anything that broke):

  http://lists.busybox.net/pipermail/uclibc/2006-July/016032.html

I used to prod other projects to support uClibc too. For example, it
took me three years to finally convince qemu to stop being incompatible
with uClibc static binaries on powerpc, but they finally made the change:

  https://lists.gnu.org/archive/html/qemu-devel/2010-02/msg00800.html

Circa 2010 I was still trying to convince people uClibc would recover
when everybody _else_ had declared it dead:

  http://lists.busybox.net/pipermail/uclibc/2010-April/043835.html

Buildroot itself was finally ready to declare uClibc dead a couple years
ago:

  http://lists.busybox.net/pipermail/buildroot/2014-February/089789.html

Which is when you stepped in to continue beating the dead horse, so they
didn't have to decide.

It's nice that you're maintaining buildroot's uClibc so they don't have
to maintain their own fork anymore (like emcraft still does, or
https://github.com/mickael-guene/uclibc/tree/uClibc-0.9.33.2-fdpic-m).
Your version is more interesting to me than random other attempt du jour
like https://github.com/davidgfnet/uClibc-Os because buildroot uses your
version.

But cortex-m still only supports pthreads in 2016 and even that's buggy
in ways that were fixed out of tree quite a while ago. The release I
fished this bugfix out of is a year old. I don't have their source
control to see how old the fix really is, but emcraft's "preferred"
kernel (https://github.com/emcraftsystems/linux-emcraft) forked off from
mainline 7 years ago, and cortex-m support for Linux is their core
business, so there's a guess how long ago somebody actually _using_ this
might have noticed it.

In case you really _don't_ know the history, let me walk you through how
the uClibc project died, going back through about ten years of
accumulated scar tissue, and why three different maintainers before you
failed to fix it. The most prominent reasons were a classic "project
tumor", an absentee maintainer, development forking itself to death, and
the code itself forking to death (most prominently over NPTL).

--- Project Tumor

The first problem is that buildroot forked off and sucked away all
uClibc's developers for several years. This doesn't mean buildroot is
bad, QEMU similarly forked off from tinycc and sucked away all that
project's developers. QEMU started life with the purpose of running Wine
on non-x86 platforms
(https://www.winehq.org/pipermail/wine-devel/2003-March/015577.html).

I usually summarize buildroot as "a test harness for uClibc that grew
legs" but the actual history is more complciated, it seems to have been
inspired by a build system for the "tuxscreen" project
(http://lists.busybox.net/pipermail/uclibc/2002-February/002542.html)
which Erik turned into a toolchain builder for uClibc
(http://lists.busybox.net/pipermail/uclibc/2002-August/024891.html when
he abandoned the old toolchain wrapper because of libgcc_s.so issues)
then added the ability to do smoketest boots under User Mode Linux
(which seems to have been brought up at
http://lists.busybox.net/pipermail/busybox/2001-April/037259.html) and
so on. For years its main purpose was people would go "How do I build X
with uClibc" and he'd go
http://lists.busybox.net/pipermail/uclibc/2002-August/004089.html

The problem is, buildroot had no obvious project boundaries (you
couldn't say "no, this is not part of buildroot") so it expanded and
expanded, turning into a Linux distro which took up more and more of the
uClibc developers' time. Erik publicly complained that this was
negatively impacting uClibc development over a decade ago:

  http://lists.busybox.net/pipermail/uclibc/2003-August/027564.html

I don't participate in buildroot development much (yes I still owe you
guys a new toybox patch) but I was there from the beginning, as in I
created the first buildroot config file
(http://lists.busybox.net/pipermail/uclibc/2003-August/027559.html) and
documentation
(http://lists.busybox.net/pipermail/uclibc/2003-August/027531.html).

It was also obvious to _me_ that buildroot traffic was drowning out
uClibc development back at the start:

  http://lists.busybox.net/pipermail/uclibc/2003-November/028342.html

But I didn't have the ability to fix it until Erik handed off busybox
maintainership to me and thus gave me root access to the shared server,
and I abused that access to create this buildroot list, the first post
in the archive of which is:

  https://lists.busybox.net/pipermail/buildroot/2006-July/012219.html

And then I tried to politely shunt the buildroot traffic off the uClibc
list:

  https://lists.busybox.net/pipermail/uclibc/2006-July/015977.html

But that was after buildroot traffic had drowned out uClibc development
traffic for THREE YEARS. Needless to say, this hurt the project's
development. Here are a few highlights from the uClibc thread that led
_up_ to me creating the buildroot list:

  http://lists.busybox.net/pipermail/uclibc/2005-October/033703.html
  http://lists.busybox.net/pipermail/uclibc/2005-October/033709.html
  http://lists.busybox.net/pipermail/uclibc/2005-October/033712.html
  http://lists.busybox.net/pipermail/uclibc/2005-October/033720.html

I.E. my motivation was that buildroot was smothering uClibc, but by the
time I was in a position to separate the two uClibc development had been
significantly eroded.

--- Absentee maintainer

The second problem uClibc died from was an absentee maintainer: Erik ran
a struggling startup that took so much of his time it impacted his
marriage. The prominent developers under him didn't fare much better,
buildroot or no: Manuel Nova was never the same after his fiancee died
(you may notice the various "donate money in memory of" headers in the
code, he got really _bitter_ as time went on and eventually wandered
off). Glenn McGrath burned out and left uClibc and BusyBox due to trying
his hand at license enforcement and having a horrible experience with it
(ala http://lists.busybox.net/pipermail/busybox/2005-April/048256.html
which was one of the things I cited when starting my own license
enforcement effort a few years alter, to take the burden off other
people trying to do it themselves). SJ Hill got mercenary and refused to
release his NPTL work until his employer paid outstanding invoices
(which dragged on for _years_ but we'll get to that).

I took busybox off of Erik's hands in part so he'd have more time to
spend on uClibc (and also so busybox didn't go the _way_ of uClibc):

  http://lists.busybox.net/pipermail/busybox/2005-January/047510.html
  http://lists.busybox.net/pipermail/busybox/2005-January/047601.html
  http://lists.busybox.net/pipermail/busybox/2005-May/048481.html
  http://lists.busybox.net/pipermail/busybox/2005-May/048565.html

And so on scaling up to me taking over busybox maintainership. (Alas the
web archive has some holes in it, ala
http://lists.busybox.net/pipermail/busybox/2005-June/048965.html). But
to clarify: I took work off or Erik's plate when he _asked_ for help
over in busybox-land:

  http://lists.busybox.net/pipermail/busybox/2005-April/048250.html
  http://lists.busybox.net/pipermail/busybox/2005-April/048255.html

And it took up all my spare time. In May 2005 I posted to the busybox
mailing list 104 times.

  http://lists.busybox.net/pipermail/busybox/2005-May/date.html

I didn't have TIME to do the same for uClibc, but I still poked at
various other issues such as the fact there was no official place to get
kernel headers from when building a C library (the kernel guys said that
coming up with sanitized kernel headers for use by userspace was a
distro maintainers' problem, hence
http://lists.busybox.net/pipermail/uclibc/2004-June/030119.html and
http://lists.busybox.net/pipermail/uclibc/2006-February/035260.html and
http://lists.pld-linux.org/mailman/pipermail/llh-discuss/2006-March/000016.html
and https://lwn.net/Articles/244375/ and so on).

Those sort of things reduced the number of people building their own
uClibc systems, outside of something like buildroot that does it for
you. Use of the project actually _declined_ as it got harder to make it
work. (Side note: the eglibc project was created by developers
commenting on how the death of uClibc made their project necessary.
Wanna know how long ago that was?)

Having chosen to get behind busybox and push, I didn't have a similar
amount of spare bandwidth to directly help out uClibc with. Taking
busybox off Erik's plate helped out uClibc a little bit, but it didn't
last. Erik's participation continued to decline until he eventually left
open source development entirely.

--- Forked development

Another reason the uClibc project died was it forked itself to death.
Ten years ago a guy named sjhill did the first uClibc NPTL port:

  https://www.kernel.org/doc/ols/2006/ols2006v1-pages-409-420.pdf

But he refused to release the code, first for various trivial reasons
(ala http://lists.busybox.net/pipermail/uclibc/2006-March/035697.html)
then he wanted to wait until after his OLS presentation, then he wanted
to wait until whoever had sponsored the work (I forget) paid him more
for it... His reasons for not merging it kept changing, and he kept
dangling the idea of merging it but never doing so for years. Here's me
declaring his work dead in an IRC conversation a couple years later
after years of it never going in:

  http://infobot.rikers.org/%23uclibc/20080810.html.gz

Other developers started maintaining out of tree forks as a matter of
course, and complaining that merging code into mainline was
inconveniencing their forks. Doing this eventually chased away the most
active remaining developer who _was_ merging code into mainline,
convincing him that _his_ fork should be out of tree too:

  http://lists.busybox.net/pipermail/uclibc/2006-March/015014.html

In the ensuing discussion other prominent developers admitted that they
valued their private forks more than mainline. Manuel Nova said, and I
quote, "I can't ethicly (at least in my code of ethics) justify handing
out bug fixes to my employer's competitors until necessary."

  http://lists.busybox.net/pipermail/uclibc/2006-March/015018.html
  http://lists.busybox.net/pipermail/uclibc/2006-April/015077.html
  http://lists.busybox.net/pipermail/uclibc/2006-April/015080.html

Here's sjhill calling me a hypocrite for saying code _should_ go upstream:

  http://lists.busybox.net/pipermail/uclibc/2006-April/015103.html

I participated fairly extensively in that thread:

  http://lists.busybox.net/pipermail/uclibc/2006-April/015083.html
  http://lists.busybox.net/pipermail/uclibc/2006-April/015097.html
  http://lists.busybox.net/pipermail/uclibc/2006-April/015127.html

But the maintainer, Erik, didn't. Half his response in that thread was
to propose removing architectures to reduce the maintenance burden. No
really:

  http://lists.busybox.net/pipermail/uclibc/2006-April/015195.html

Erik did not object to the project forking itself to death, specifically
the "we must prevent code going upstream to not inconvenience these out
of tree forks" thread topic was fine by him. I objected to his lack of
objection:

  http://lists.busybox.net/pipermail/uclibc/2006-April/015085.html

But it wasn't _my_ project, and I couldn't overrule its maintainer when
he didn't  care to prevent it from forking itself to death.

--- Forked code

Development wasn't the only thing that fragmented, the in-tree codebase
did too. A persistent habit of uClibc's was cloning large chunks of
glibc. Since the two projects were under the same license, the uClibc
developers could copy a glibc subsystem (such as pthreads) more or less
verbatim, and then try to strip it down to be smaller afterwards. Their
snapshot would inevitably fall out of date with glibc's, and rather than
try to update it themselves they'd eventually clone a less stale version
and put it next to the other one in the tree with a menuconfig option to
select the "old" or "new" version. On more than one occasion they had
three snapshots of the same infrastructure in play at once.

Here's an incomplete list of some of the duplicate infrastructure you
had to select old or new versions of in menuconfig:

  http://lists.busybox.net/pipermail/uclibc/2008-December/041639.html

Just educating yourself on what your OPTIONS were was a daunting task.
And the menuconfig options and help text were often completely
nonsensical. I used to go through and try to audit menuconfig:

  http://lists.busybox.net/pipermail/uclibc/2008-August/040766.html
  http://lists.busybox.net/pipermail/uclibc/2009-August/042851.html

But tended to get pushback from people going "no, it makes sense to _me_
and I refuse to let it be changed":

  http://lists.busybox.net/pipermail/uclibc/2009-August/042868.html

Needing menuconfig at all is another design problem: configuring uClibc
is not a trivial exercise, and all sorts of configuration changes change
the ABI and break binary compatibility. (And of course no uClibc version
upgrade was ever binary compatible with any previous version.) You don't
have this problem with musl-libc, not only _is_ there no menuconfig
stage, but they tried for binary compatibility with glibc (as in trying
to make it a drop-in replacement capable of running the flash plugin and
such) for the longest time, and thus musl has a reasonably stable ABI.
(It's not 100% glibc compatible but they know what their ABI _is_.)

But back to duplicate code: different architectures even had their own
versions of headers that _should_ have been mostly the same, but were
full of version skew. Cleaning this up was a perennial problem, with
occasional small victories:

  http://lists.busybox.net/pipermail/uclibc-cvs/2010-April/027938.html

But mostly, wrestling with this burned out developers and sucked up huge
amounts of effort while falling behind the rate of increase in clutter
and scar tissue in the code (or else the code going stale and not having
current features), while introducing churn and breakage into the
development cycle that exacerbated the lack of releases (because if such
a cleanup broke something nobody would notice for a year or more; the
churn made releases dangerous to do, so the developers simply avoided
having any).

The worst instance of redundant code was uClibc's NPTL support, or lack
thereof. In Linux 2.6 Rusty Russell invented Futexes and the "New Posix
Threading Library" was invented on top of that circa 2004.


http://www.drdobbs.com/open-source/nptl-the-new-implementation-of-threads-f/184406204

The first uClibc version of NPTL support was the aforementioned work by
sjhill, who didn't merge it upstream. As a result, several _other_ NPTL
implementations were done for other architectures, each starting from
scratch and sharing zero code with any of the others:

  http://lists.busybox.net/pipermail/uclibc/2006-March/014793.html
  http://lists.busybox.net/pipermail/uclibc/2006-December/037765.html
  http://lists.busybox.net/pipermail/uclibc/2007-August/039206.html

This combined with the earlier "don't destabilize my out of tree fork by
merging anything upstream" nonsense to ensure that all the different
NPTL implementations stayed as out-of-tree forks for years and years,
slowly diverging, because there was no way to get them upstream. Merging
any of them would inconvenience the others! No, they had to be unified
_before_ they could be merged, which was a chicken and egg problem.

This led to at least four complete and active NPTL ports done by four
groups of people for four different architectures (arm, mips, powerpc,
sh), which had no common code. Multiple summit-like meetings were held
online to try to deal with this, but we still had multiple versions of
the old pthreads code NPTL was supposed to replace, and couldn't even
agree on unifying or removing _those_ duplicates:

  http://lists.busybox.net/pipermail/uclibc/2008-September/041039.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020198.html

So the situation dragged on for years and years and the code grew more
and more scar tissue as everybody's cleanup efforts focused on their own
fork and not mainline. When releases _did_ happen, they were just a
subset of the development, major features like NPTL were kicked down the
road again and again, in various deveopment branches but not in any
release version.

--- _TWO_ new maintainers didn't help

Circa 2007 uClibc was so clearly dying I stopped being polite about it.

By this point the project was a giant mass of scar tissue from partial
merges of various developers' out of tree forks leaving half-finished
code all over the tree. Nobody could get major subsystems like locales
to work at all, there were multiple regex implementations... Unifying
that mess was a giant cleanup job that nobody had the bandwidth to do,
and the longer it went on the bigger a job it became. Endless _partial_
cleanups just made the problem _worse_, the churn destabilizing the tree
and making releases harder to do.

Along the way Erik basically stopped participating at all. So a year or
so after he handed BusyBox off to me, I poked him to hand off uClibc
maintainership to somebody and just do Buildroot. (We were talking a lot
behind the scenes because of the GPL enforcement suits, which seemed
like a good idea at the time but wound up drawing Erik further away from
day-to-day programming).

But the new uClibc maintainer Erik chose was Mike Frysinger, who
maintained a prominent out of tree fork for blackfin (proprietary, only
customers could access it). And over the next year, Mike never put out a
release. He allowed the gap since the last release to grow to its
highest level ever, and then stopped posting to the uClibc list entirely
to focus on his proprietary blackfin port (which was only available to
customers). When it had been FOUR MONTHS since Mike last posted to the
list, while everybody else was trying to get a release together (because
hey, new maintainer would unblock the project, right?)

So after the new maintainer didn't cut a release for a year and didn't
post to the list for four motnhs straight, I got sick of him:

  http://lists.busybox.net/pipermail/uclibc/2008-September/019987.html

And basically staged a coup and appointed a new maintainer who had the
benefit of actually SHOWING UP:

  http://lists.busybox.net/pipermail/uclibc/2008-October/020339.html

(Remember, root access to the server, and the previous maintainer
answered my email.)

I pitched in and started testing everything, and sending in patches to
try to get a release out and get development unstuck under the new
maintainer (Bernhard, who is still the current nominal maintainer of
uClibc, I.E. the guy _you_ stepped in to replace):

  http://lists.busybox.net/pipermail/uclibc/2008-September/020033.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020178.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020182.html
  http://lists.busybox.net/pipermail/uclibc/2008-September/020180.html

Of course bernhard's last release was in 2012, and he let things go over
three years after that without a release before I basically gave up,
stopped performing CPR, and called it. The uClibc project was dead.

Presumably you know it from there, but you're now the FOURTH maintainer
of the project _since_ it died.

You came into a project I'd pushed for 10 years and started collecting
trivial bugfixes without addressing any of the serious architectural
issues (like needlessly duplicated per-architecture headers snapshot
from ancient glibc versions, with a #define pretending to be glibc (musl
doesn't) and declaring all sorts of stuff the library doesn't actually
implement). And you're wondering why I have a more pessimistic outlook
of your chances than you do?

The big elephant in the development room was the NPTL code, and cortex-m
is still using pthreads, not nptl, and WITHIN that pthreads mess is _old
and _new versions of thread wait for supporting 2.2 kernels. (The
menuconfig option has been replaced with _old and _new suffixes on the
functions. Great.)

That does not say to me that you've actually _fixed_ anything. Sure you
can maintain your own version. Just like BeOS isn't really dead as long
as the Haiku project persists, or osfree.org is keeping OS/2 alive, or
reactos can keep windows 95 alive, the multiple open source AmigaOS
clone attempts. But I'm not sure why I should CARE.

You've said that your reason for supporting uclibc-ng was for two
architectures, ARC and Xtensa:

  http://www.openwall.com/lists/musl/2015/05/30/1

That is why _you_ care. I personally think that adding support for those
to musl-libc would be a far better use of your time, but it's not my job
to tell you want to do. It's your hobby time. Do what you like with it.
Just don't expect me to take you seriously after ten years of this.

> It is really cool and very nice that the communication between Rich
> and me is always fruitful. I always report any bugs I find while
> testing musl on new or old platforms (mostly via IRC channel) and I
> always take care that bugfixes for musl targets end up in buildroot.
> (mipsr6 support, binutils microblaze problems, ..)
> 
> I really would like to see a similar open communication between the 
> nommu community, you and me. 
> 
> Just ignoring uClibc-ng does not will make it die.

I cc'd buildroot. That is a real project, which still uses uClibc for
the targets Rich hasn't gotten around to porting musl to yet. I cc'd
them so they would be aware of the issue. I could have just sent it to
the musl list, but chose to inform buildroot as well.

(Oddly enough, lists.buildroot.org doesn't have a web archive of the
list. I have to go to lists.busybox.net to see the archive? I would have
thought they'd moved it over by now. I can _email_ @buildroot...)

> As a good starting point, a nice bug report and/or test application
> which allows me to reproduce the problem would be really
> appreciated.

I spent several days trying to come up with a decent test case, it was
Jonathan finding the eratta that says "this particular instruction has
to get interrupted" that let me know what was actually going on (and
that a reliable reproduction sequence can't be isolated, you need
something like the serial port going to hammer the system with
interrupts to make it happen in under an hour).

That said, the board crashing under load is pretty easy to spot.
Figuring out _why_ was the hard part. :)

Of course this particular group of people using uClibc for a new
cortex-m project were originally specced to use "uclinux". As in
http://www.uclinux.org/ which had its a total disk failure take out its
CVS repository a few years back, but one guy is still putting out
releases every year by editing the previous year's release files, with
no source control. Therefore the project still isn't dead despite the
newest file in the "http download" link on the left being from 2003 and
their "the developers" page having a... ahem, _slightly_younger_ picture
of Jeff Dionne than https://lwn.net/Articles/647636/ (who moved to Japan
in 2003 and hasn't particularly been involved with that project since
then)...

I pointed them at buildroot instead, and I informed buildroot of a bug
using their stuff. If uclinux wants to complain I didn't cc: them on
this bug either...

> best regards
>  Waldemar

Rob

Re: [musl] cortex-m support?

[Thomas Petazzoni]

Hello,

On Tue, 20 Dec 2016 01:18:40 -0600, Rob Landley wrote:

> I cc'd the buildroot list, which only has uClibc-based cortex-m support
> at the moment. Why do you suppose I did that?
> 
> Did you want me to send it to the uclibc.org mailing list which hasn't
> had a single post this month except your announcement of your fork's
> release? The list where nobody's noticed the chrome browser can't access
> https://lists.uclibc.org (archives, subscription page, etc) for weeks
> now? And yes, I publicized that fact when I noticed it:

Do you realize that the uclibc-ng project has a mailing list? It is
active, people post patches, and they get merged.

I posted patches on this list, they were merged by Waldemar within the
next day or two.

> Your fork clearly hasn't fixed any of the structural issues uClibc
> developed over the years.

Waldemar has fixed the main problem of uClibc: the lack of regular
releases, the lack of a responsive maintainer that merges patches.

Then, from a technical point of view, Waldemar has added new features,
dropped badly supported architectures, cleaned up a lot of things,
improved the test suite, and more. There are probably a tons of other
things to improve in uClibc-ng, but it's just a matter of receiving
contributions: we can no longer blame the lack of maintainership.

[... snip the rest of the text, TLDR ... ]

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [musl] cortex-m support?

[Rob Landley]

On 12/20/2016 02:26 AM, Thomas Petazzoni wrote:
> Hello,
> 
> On Tue, 20 Dec 2016 01:18:40 -0600, Rob Landley wrote:
> 
>> I cc'd the buildroot list, which only has uClibc-based cortex-m support
>> at the moment. Why do you suppose I did that?
>>
>> Did you want me to send it to the uclibc.org mailing list which hasn't
>> had a single post this month except your announcement of your fork's
>> release? The list where nobody's noticed the chrome browser can't access
>> https://lists.uclibc.org (archives, subscription page, etc) for weeks
>> now? And yes, I publicized that fact when I noticed it:
> 
> Do you realize that the uclibc-ng project has a mailing list?

No, I hadn't noticed. Good to know, I guess?

>> Your fork clearly hasn't fixed any of the structural issues uClibc
>> developed over the years.
> 
> Waldemar has fixed the main problem of uClibc: the lack of regular
> releases, the lack of a responsive maintainer that merges patches.

Bernhard was pretty active in the years right after he became
maintainer. In the long run 10 years of accumulated technical debt was
too much for him, but maybe it'll be different this time.

> Then, from a technical point of view, Waldemar has added new features,
> dropped badly supported architectures, cleaned up a lot of things,
> improved the test suite, and more. There are probably a tons of other
> things to improve in uClibc-ng, but it's just a matter of receiving
> contributions: we can no longer blame the lack of maintainership.

Good luck to him. I have nothing against uclibc-ng the same way I have
nothing against olibc. If I wasn't already regression testing against
glibc, bionic, and musl, I might pay attention to this one.

> [... snip the rest of the text, TLDR ... ]

He asked. :)

Rob

Re: [musl] Re: [musl] cortex-m support?

[Waldemar Brodkorb]

Hi Rob,
Rob Landley wrote,

> > I am wondering why you don't cc me or any uclibc related list? 
> 
> I cc'd the buildroot list, which only has uClibc-based cortex-m support
> at the moment. Why do you suppose I did that?

That is not completely true, OpenADK has support for it, too.
But I am not sure you declare OpenADK a real project.
It seems to me that in your definition of a open source project,
the people behind must either get money for their work or the
project needs millions of installations.
 
> Did you want me to send it to the uclibc.org mailing list which hasn't
> had a single post this month except your announcement of your fork's
> release? 

No, I would like to see your suggestions and patches on
the uClibc-ng mailing list:
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel/

> Has your fork solved the locales problem?
> 
>   http://lists.busybox.net/pipermail/uclibc/2015-June/049000.html

I don't support the binary locale package. I am not very interested
in locale support.
 
> Has your fork solved the nptl issue?
> 
>   http://lists.busybox.net/pipermail/uclibc/2008-September/020151.html
>   http://lists.busybox.net/pipermail/uclibc/2008-September/020169.html
>   http://lists.busybox.net/pipermail/uclibc/2008-September/020171.html
>   http://lists.busybox.net/pipermail/uclibc/2008-October/041201.html

Not sure. But in uClibc-ng there is Linuxthreads support for every
supported architecture (excluding METAG) and NPTL for the global
players. I added NPTL support from GNU libc recently, but Microblaze
seems indeed bitrotted inside GNU libc as nobody is doing any test
suite runs. 
https://sourceware.org/ml/libc-alpha/2016-07/msg00640.html

There is even missing DWARF2 support in GCC upstream, so I think it
is even not compile time tested.
 
>   http://lists.busybox.net/pipermail/uclibc/2006-March/014811.html
> 
> Do you have a sane "make defconfig" that lets people build uClibc
> without learning what over a hundred individual config options do and
> making decisions about whether or not they need each one? This issue
> doesn't even come _up_ with musl, it fundamentally avoids most of the
> structural problems that strangled uClibc development, by design.

I regulary cleanup needless options and keep good defaults.
The number of options get lower. I still like the idea of being able
to build a complete toolchain without thread support or other
stuff disabled. I will try to remove more options in the future.
 
> > You still believe uClibc projects should die?
> 
> No, I believe uClibc _already_ died. I believe this because I was there.

Yes, uClibc is dead. uClibc-ng is alive!
 
> But cortex-m still only supports pthreads in 2016 and even that's buggy
> in ways that were fixed out of tree quite a while ago. The release I
> fished this bugfix out of is a year old. I don't have their source
> control to see how old the fix really is, but emcraft's "preferred"
> kernel (https://github.com/emcraftsystems/linux-emcraft) forked off from
> mainline 7 years ago, and cortex-m support for Linux is their core
> business, so there's a guess how long ago somebody actually _using_ this
> might have noticed it.

So, may be you can prepare a patch for the uClibc-ng list?
Or do you want to see the bug open for another year? 
 
> In case you really _don't_ know the history, let me walk you through how
> the uClibc project died, going back through about ten years of
> accumulated scar tissue, and why three different maintainers before you
> failed to fix it. 

Thanks for the history walkthrough. Do you know why Bernhard is so
unresponsive and more about the time shortly after his last release?

> You came into a project I'd pushed for 10 years and started collecting
> trivial bugfixes without addressing any of the serious architectural
> issues (like needlessly duplicated per-architecture headers snapshot
> from ancient glibc versions, with a #define pretending to be glibc (musl
> doesn't) and declaring all sorts of stuff the library doesn't actually
> implement). And you're wondering why I have a more pessimistic outlook
> of your chances than you do?

You have to start somewhere, otherwise nothing happens.
 
> The big elephant in the development room was the NPTL code, and cortex-m
> is still using pthreads, not nptl, and WITHIN that pthreads mess is _old
> and _new versions of thread wait for supporting 2.2 kernels. (The
> menuconfig option has been replaced with _old and _new suffixes on the
> functions. Great.)

Linuxthreads.new is gone. uClibc-ng only support Linuxthreads and
NPTL.

> You've said that your reason for supporting uclibc-ng was for two
> architectures, ARC and Xtensa:
> 
>   http://www.openwall.com/lists/musl/2015/05/30/1
 
> That is why _you_ care. I personally think that adding support for those
> to musl-libc would be a far better use of your time, but it's not my job
> to tell you want to do. It's your hobby time. Do what you like with it.
> Just don't expect me to take you seriously after ten years of this.

That is not entirely right. I care for every supported uClibc-ng
architecture. uClibc-ng might be used for different use cases:
- old non mainstream architectures like lm32, avr32, cris, fr-v, ..
- noMMU systems like arm, coldfire, bfin, c6x, ..
- architectures not supported by any other C library like arc,
  xtensa, nds32, h8300, ..
- vintage unix hardware like SGI O2, Sparcstations, ...

See my table which architectures are not supported by musl/glibc:
http://uclibc-ng.org/wiki/matrix

Indeed Max has started a musl implementaton for Xtensa:
https://github.com/jcmvbkbc/musl-xtensa

The Synopsys people trying to add support for GNU libc.

But even then uClibc-ng is always a good choice for regression
testing and testing between the different C libraries.
 
> I cc'd buildroot. That is a real project, which still uses uClibc for
> the targets Rich hasn't gotten around to porting musl to yet. I cc'd
> them so they would be aware of the issue. I could have just sent it to
> the musl list, but chose to inform buildroot as well.

Other real projects use uClibc-ng, too. So may be next time you sent
it to our list directly ;)
 
best regards
 Waldemar

Re: [musl] Re: [musl] cortex-m support?

[Rob Landley]

On 12/21/2016 12:18 AM, Waldemar Brodkorb wrote:
> Hi Rob,
> Rob Landley wrote,

I missed this the first time because gmail is crap. (I wash email
through it for the spam filtering, but there's no way to get it to STOP
duplicate killing, so a reply like this gets randomly sorted into the
buildroot folder, the musl folder, or my inbox depending on which copy
arrvied first. Makes following threads impossible, let alone archiving
them. Yes I have complained to Google multiple times over the past few
years, they're too big to hear me.)

So I just noticed it in the musl web archive, but I think it's off topic
here. I assume your list will bounce posts from non-subscribers, so
lemme send my reply to the old uClibc mailing list instead. (*shrug*
Still subscribed there, and you're the only one who posts there, so...)

Rob

Re: cortex-m support?

[Rich Felker]

On Thu, Dec 15, 2016 at 12:34:00PM -0600, Rob Landley wrote:
> On 12/08/2016 03:11 PM, Rich Felker wrote:
> > On Tue, Dec 06, 2016 at 11:52:29PM -0600, Rob Landley wrote:
> >> Index: src/setjmp/arm/setjmp.S
> >> ===================================================================
> >> --- src/setjmp/arm/setjmp.S	(revision 0)
> >> +++ src/setjmp/arm/setjmp.S	(revision 4920)
> >> @@ -0,0 +1,45 @@
> >> +.global __setjmp
> >> +.global _setjmp
> >> +.global setjmp
> >> +.type __setjmp,%function
> >> +.type _setjmp,%function
> >> +.type setjmp,%function
> >> +__setjmp:
> >> +_setjmp:
> >> +setjmp:
> >> +	mov ip,r0
> >> +#if defined(__thumb__)
> >> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,lr}
> >> +        str sp, [ip], #4
> >> +#else
> >> +	stmia ip!,{v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
> >> +#endif
> > 
> > My leaning is to just always use the version that works on thumb,
> 
> Presumaby there's a longjmp.S bit too because:
> 
>         ldmia ip!, {v1,v2,v3,v4,v5,v6,sl,fp,sp,lr}
> 
> Make sure you're aware of erratum 752419:
> 
>   https://sourceware.org/ml/binutils/2011-05/msg00087.html

Thanks. I wasn't aware of this. But my proposed version of the patch
is already immune; it doesn't use ldr to sp, but instead ldm to a temp
register followed by mov to sp. I grepped for ldr to sp and did not
find any instances that are incrementing forms.

Rich