mailing list of musl libc
 help / color / mirror / code / Atom feed
* Moving forward with sh2/nommu
@ 2015-06-01 15:11 Rich Felker
  2015-06-02  6:09 ` Rob Landley
  2015-06-10  3:30 ` Rich Felker
  0 siblings, 2 replies; 17+ messages in thread
From: Rich Felker @ 2015-06-01 15:11 UTC (permalink / raw)
  To: musl; +Cc: rob

[resent to musl list]

Here's a summary of the issues we need to work through to get a modern
SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
and to the point where it's something people can use roughly 'out of
the box':

Kernel issues:

1. Kernel should support loading plain ELF directly, unmodified. Right
   now I'm writing 0x81 to byte 38 of the header to make a "non-FDPIC
   FDPIC ELF binary", which works, but I have to make a personality()
   syscall at startup to switch back (this matters to kernel signal
   handling) and it's just highly inconvenient/ugly.

   Despite plain ELF being suitable for NOMMU, the loader
   implementation in binfmt_elf.c depends pretty heavily on MMU. The
   one in binfmt_elf_fdpic.c can work on either. The easiest way
   forward is to make it so that binfmt_elf_fdpic.c does not insist on
   having the FDPIC flags in the ELF header on NOMMU targets (where it
   won't confict with binfmt_elf.c since that loader isn't usable).

2. Kernel insists on having a stack size set in the PT_GNU_STACK
   program header; if it's 0 (the default ld produces) then execve
   fails. It should just provide a default, probably 128k (equal to
   MMU-ful Linux).

3. Kernel uses the stack for brk too, growing brk from the opposite
   end. This is horribly buggy/dangerous. Just dummying out brk to
   always-fail is what should be done, but if it can't be done on the
   kernel side musl can do it instead (that's what I'm doing now).
   Unfortunately I suspect fixing this might be controversial since
   there may be existing binaries using brk that can't fall back to
   mmap.

4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
   is that these two SH2A hardware traps overlap with the syscall
   range used by SH3/4 ABI:

	#  define TRAP_DIVZERO_ERROR  17
	#  define TRAP_DIVOVF_ERROR   18

   The path forward I'd like to see is deprecating everything but trap
   numbers 22 and 38, which, as far as I can tell, are safe for both
   the SH2 and SH3/4 kernel to treat as syscalls. These numbers
   indicate "6 arguments"; there is no good reason to encode the
   number of arguments in the trap number, so we might as well just
   always use the "6 argument" code which is what the variadic
   syscall() has to use anyway. User code should then aim to use the
   correct value (22 or 38) for the model it's running on (SH3/4 or
   SH2) for compatibility with old kernels, but will still run safely
   on new kernels if it detects wrong.

Toolchain issues:

1. We need static-PIE (with or without TEXTRELs) which gcc does not
   support out of the box. I have complex command lines that produce
   static-PIE, and I have specfile based recipes to convert a normal
   toolchain to produce (either optionally or by default) static-PIE,
   but these recipes conflict with using the same toolchain to build
   the kernel. If static-PIE were integrated properly upstream that
   would not be an issue.

2. Neither binutils nor gcc accepts "sh2eb-linux" as a target. Trying
   to hack it in got me a little-endian toolchain. I'm currently just
   using "sheb" and -m2 to use sh2 instructions that aren't in sh1.

3. The complex math functions cause ICE in all gcc versions I've tried
   targetting SH2. For now we can just remove src/complex from musl,
   but that's a hack. The cause of this bug needs to be found and
   fixed in GCC.

musl issues:

1. We need runtime detection for the right trap number to use for
   syscalls. Right now I've got the trap numbers hard-coded for SH2 in
   my local tree.

2. We need additional runtime detection options for atomics: interrupt
   masking for plain SH2, and the new CAS instruction for SH2J.

3. We need sh/vfork.s since the default vfork.c just uses fork, which
   won't work. I have a version locally but it doesn't make sense to
   commit without runtime trap number selection.

4. As long as we're using the FDPIC ELF header flag to get
   binfmt_elf_fdpic.c to load binaries, the startup code needs to call
   the personality() syscall to switch back. I have a local hack for
   doing this in rcrt1.o which is probably not worth upstreaming if we
   can just make the kernel do it right.

5. The brk workaround I'm doing now can't be upstreamed without a
   reliable runtime way to distinguish nommu. To put it in malloc.c
   this would have to be a cross-arch solution. What might make more
   sense is putting it in syscall_arch.h for sh, where we already
   have to check for SH2 to determine the right trap number; the
   inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-01 15:11 Moving forward with sh2/nommu Rich Felker
@ 2015-06-02  6:09 ` Rob Landley
  2015-06-02 16:45   ` Rich Felker
  2015-06-10  3:30 ` Rich Felker
  1 sibling, 1 reply; 17+ messages in thread
From: Rob Landley @ 2015-06-02  6:09 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl

On Mon, Jun 1, 2015 at 11:04 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Mon, Jun 01, 2015 at 01:19:32AM -0500, Rob Landley wrote:
>> FYI, Jeff's response.
>>
>> We REALLY need to get this on the mailing list.
>>
>> Rob
>
> OK, done.

Actually I was hoping you and jeff could repost your respective bits, but eh.


>> > 2. Kernel insists on having a stack size set in the PT_GNU_STACK
>> >   program header; if it's 0 (the default ld produces) then execve
>> >   fails. It should just provide a default, probably 128k (equal to
>> >   MMU-ful Linux).

MMU-ful linux preallocates 0k and then demand faults in pages. MMU-ful
almost never has to worry about memory fragmentation because it can
remap _and_ move physical pages around (if nothing else, evict through
swap).

This is dedicated contiguous allocation even if it's wasted in a
system that's _very_ prone to fragmentation, meaning things like
"true" can fail well before you're in OOM killer territory.

It's not the same at all.

>> Nooooo.  8k.  uClinux programs cannot depend on a huge stack, because that
>> means each instance needs to kmalloc() a huge block of memory.  That is
>> bad, but it leads to failure to load because of fragmentation (not being
>> able to find contiguous memory blocks for all those stacks).
>
> My view here was just that the default, which none was specified while
> building the program, should be something "safe". Failed execve
> ("oops, need to use the right -Wl,-z,stack-size=XXX") is a lot easier
> to diagnose than a stack overflow that clobbers the program code with
> stack objects. Right now the default is "always fails to load" because
> the kernel explicitly rejects any request for a default.

I note that Rich was probably saying he wants the default at 128k for
ELF, not for FDPIC. That said, I'm not sure you can have a big enough
warning sign about vanilla elf being crappy in that case.

There's 2 things to balance here: if it doesn't "just work" then
people are frustrated getting their programs to run, but if the
defaults are horrible for scalability people will go "nommu is crap,
we can't use it" without ever learning what's actually _wrong_. (It's
a lot easier to get people to fix obvious breakage than to performance
tune something that becomes untenable after the fact. How big the
performance hit has to be before requiring --no-i-really-mean-it on
the command line is an open question, but this is up there.)

Of the two ("just works" but is horrible, breaks for trivial things
until you understand why), if they can't get "hello world" to work we
can probably get them to read a very small HOWTO, so they at least
know fixed stack size is an _issue_ in this context. (We're already in
"fork does not work" territory. We are not in Kansas anymore, if you
try to fake kansas _too_ hard you're doing developers a disservice.)

That said, annotating every package in the build is silly. Probably
there should be an enviornment variable or something that can set this
default for entire builds, and something to actually _measure_ stack
usage after a run would be awesome. (The kernel has checkstack.pl for
example?) And the elf2flt command line option for setting stack size
was just _awkward_, no idea what you've done for your fdpic binaries
but the traditional UI for this is horrible.

>> >   Unfortunately I suspect fixing this might be controversial since
>> >   there may be existing binaries using brk that can't fall back to
>> >   mmap.
>>
>> No, look at what I did in uClibc for brk().  I think it fails, and everything
>> in the past depends on that.
>
> OK. So do you think it would be safe/acceptable to make brk always
> fail in the kernel?

Fork does, this is _less_ intrusive than that.

> As long as making it fail is left to userspace,
> the userspace code has to know at runtime whether it's running on
> nommu or not so it can make brk fail. (I'm assuming my goal of having
> binaries that can run on both/either.)

Gotta make a certain amount of historical usage work if we're to wean
people off their weird bespoke builds. Not sure the right answer here.
Then again a patch to make future kernels do this right and then
depending on that is a problem that will solve itself in time. (We no
longer care about 2.4. I've generally used 7 years as a rule of thumb
for "that's too old to care about without a reason", and that gets us
back to around 2.6.25 at the moment...)

>> > 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
>> >   is that these two SH2A hardware traps overlap with the syscall
>> >   range used by SH3/4 ABI:
>> >
>> >        #  define TRAP_DIVZERO_ERROR  17
>> >        #  define TRAP_DIVOVF_ERROR   18
>>
>> No, 2A is actually the -newest- SH.  This is just gratuitous breakage, and it’s
>> really unfortunate.
>>
>> >   The path forward I'd like to see is deprecating everything but trap
>> >   numbers 22 and 38, which, as far as I can tell, are safe for both
>> >   the SH2 and SH3/4 kernel to treat as sys calls.
>>
>> Kawasaki-san?  Thoughts?
>>
>> >   These numbers
>> >   indicate "6 arguments"; there is no good reason to encode the
>> >   number of arguments in the trap number, so we might as well just
>> >   always use the "6 argument" code which is what the variadic
>> >   syscall() has to use anyway. User code should then aim to use the
>> >   correct value (22 or 38) for the model it's running on (SH3/4 or
>> >   SH2) for compatibility with old kernels, but will still run safely
>> >   on new kernels if it detects wrong.
>>
>> I say drop backward compatibility.
>
> That generally goes against kernel stability principles and seems
> unlikely to be acceptable upstream.

I think Jeff means software-level backward compatibility with sh2
wouldn't be missed. (Linux wasn't big on that architecture in the
first place, and buildroot dropped support for it a release or two
back.) Losing backward compatibility with sh4 would make us look
really bad, especially since musl already supports it, but I don't
think he's suggesting breaking sh4.

If we make sh2 binaries accept sh4 syscalls, we can call it a new
architecture variant. (Which we actually _are_ with sh2j.) The kernel
patch to make that work would be tiny and the hardware doesn't change.

Given that qemu guys implemented "qemu-system-sh4" instead of
"qemu-system-sh", the current perception is that superh _is_ sh4. Not
being compatible with sh4 probably has higher overhead than not being
compatible with historical linux for sh2.

>> > Toolchain issues:
>> >
>> > 1. We need static-PIE (with or without TEXTRELs) which gcc does not
>> >   support out of the box. I have complex command lines that produce
>> >   static-PIE, and I have specfile based recipes to convert a normal
>> >   toolchain to produce (either optionally or by default) static-PIE,
>> >   but these recipes conflict with using the same toolchain to build
>> >   the kernel. If static-PIE were integrated properly upstream that
>> >   would not be an issue.
>>
>> I’d like to see -exactly- what the position independence code generator
>> is doing in all cases (there are some interesting ones).  Embedded systems
>> really do need to count every cycle, and while I’m good with making things
>> as rational and standard as possible, if the function overhead is 50 cycles
>> and blasts the iCache taking a trip through the trampoline, I think we
>> need to reconsider.  Some serious benchmarking is also in order.  bFLT does
>> not have any overhead at run time, which is why people still use it over
>> FD-PIC on a lot of platforms...
>
> There's no trampoline (I assume you mean PLT?) for static-PIE. Since
> the linker resolves all symbolic references at ld-time, it never
> generates PLT thunks; instead, all the calls using relative @PLT
> addresses turn into what you would get with @PCREL addresses. So in
> terms of calls, the main difference versus non-PIC is that you get a
> braf/bsrf with a relative address instead of a jmp/jsr with an
> absolute address, and in principle this leads to fewer relocations
> ('fixups') at runtime.
>
> However, if PIC is too expensive for other reasons, there's no
> fundamnental reason it has to be used. This is what I meant above by
> "with or without TEXTRELs". You're free to compile without -fPIE or
> -fPIC, then link as [static-]PIE, and what you'll end up with is
> runtime relocations in the .text segment. Unlike on systems with MMU,
> this is not a problem because (1) it's not shareable anyway, so you're
> not preventing sharing, and (2) there's no memory protection against
> writes to .text anyway, so you're not sacrificing memory protection.
> I believe doing it this way gets you the same results (even in terms
> of number/type of relocations) that you would get with non-shareable
> bFLT.

Poke me on irc and I'll see what I can scrounge up. (Frantically
preparing for thursday's talk about how we open sourced our code and
vhdl and documented everything by beating our stuff into uploadable
shape and documenting everything. :)

>> > 2. Neither binutils nor gcc accepts "sh2eb-linux" as a target. Trying
>> >   to hack it in got me a little-endian toolchain. I'm currently just
>> >   using "sheb" and -m2 to use sh2 instructions that aren't in sh1.
>>
>> That is why you really want to configure —target=sh2-uclinux (or make
>> sh2-linux do the right thing).  SH2 I think is always Big...
>
> Well GCC seems to consider the plain sh2-linux target as
> little-endian.

They're crazy and broken about a lot of stuff. Sega Saturn was big
endian. (Saturn was to sh2 what dreamcast was to sh4.)

I also note that gcc 4.2.1 and binutils 2.17 parsed sh2eb. Current
stuff not doing so is a regression.

> I think a lot of the issue here is that, to you, sh2 means particular
> hardware architecture, whereas to the GCC developers and to me, sh2
> means just the ISA, and from a non-kernel/non-baremetal perspective,
> just the userspace part of the ISA, which is a subset of the sh3/4
> ISAs.

Extra bit of fun:

When Renesas was implementing the ELF spec, they used a version that
had been translated into japanese. The translation program switched
codepages, meaning _ and . got swapped. (This is why the superh
prefixes are borked, the developers accurately implemented the
documentation they had.)

That said, "sh2eb-elf" and "sh2eb-unknown-linux" are different targets
with different ELF prefixes, and our ROM bootloader code was written
for the ELF one. (I looked at porting it and it's really painful and
intrustive, and non-linux code tends to be written for -elf toochains
instead of -linux toolchains in general.)

Meaning I have to build _both_ toolchains and use one for the hardware
build (which includes the ROM bootloader code) and one for the kernel
and userspace builds (we boot vmlinux and the bootloader parsing the
elf cares about the prefixes; yes the bootloader that has to build
with one set of prefixes expects to parse code built with the _other_
set, don't get me started on that).

So when you say "what gcc developers think" the answer is "they
don't". It's an inconsistent mix of random historical crap and we have
to make the best of it. I'd like to try to be the least amount of
crazy we can going forward, please.

> So when gcc is generating code for "sh2-linux", it's treating it
> as using all the usual linux-sh conventions (default endianness,
> psABI, etc.) but restricted to the sh2 instruction set (no later
> instructions).

There really _aren't_ usual "linux-sh" conventions, there's the
perception that all the world's a <strike>vax</strike> sh4 and
everybody who doesn't think that is largely still using gcc 3.4
because that never stopped working and the new stuff breaks every
third release. (Chronic problem in the embedded world, getting people
to upgrade off of what they first got working, let alone interact with
upstream via anything other than an initial smash-and-grab and
hightailing it to the hideout and then staying silent until the
statute of limitations runs out. And that's _without_ factoring a
language barrier into it.)

>> > 3. The complex math functions cause ICE in all gcc versions I've tried
>> >   targetting SH2. For now we can just remove src/complex from musl,
>> >   but that's a hack. The cause of this bug needs to be found and
>> >   fixed in GCC.

Can I get a test program I can build and try with Aboriginal's toolchain?

>> Does it happen in 4.5.2 or the Code Sorcery chain?  We need complex.
>
> I'm not sure.

He's referring to
http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu/renesas-2011.03-36-sh-uclinux.src.tar.bz2
and renesas-2011.03-36-sh-uclinux-i686-pc-linux-gnu.tar.bz2 built from
that, both of which was there last month but seems to have gone down.
Grrr. And of course mentor graphics put a robots.txt to block
archive.org because they're SUCH an open source company it exudes from
their pores.

Right, I threw both on landley.net for the moment, probably take 'em
down again this weekend. (It's GPL code and that's the corresponding
source, there you go.)

Anyway, buildroot used to use this stuff to build toolchains (ala
http://git.busybox.net/buildroot/commit/?id=29efac3c23df9431375f26d1b240627f604f42ca)
but there was serious whack-a-mole in tracking where they moved it
this week (http://git.buildroot.net/buildroot/commit/?id=27404dad33a8f9068faa8be72916ed47f905b5e6)
so...

Building from source with vanilla is _so_ much nicer...

>> > musl issues:
>> >
>> > 1. We need runtime detection for the right trap number to use for
>> >   syscalls. Right now I've got the trap numbers hard-coded for SH2 in
>> >   my local tree.
>>
>> I don’t agree.  Just rationalise it.  Why can SH3 and above not use the
>> same traps as SH2?
>
> Because the kernel syscall interface is a stable API. Even if not for
> that, unilaterally deciding to change the interface does not instill
> confidence in the architecture as a stable target.

And because the perception out there in linux-land is that sh4 was a
real (if stale) processor and sh2 wasn't, so breaking sh4 to suit sh2
_before_ we've established our new open hardware thing as actually
viable will get the door slammed on us so hard...

> OTOH if we could change to using the SH2 trap range as the default and
> just keep the old SH3/4 range as a 'backwards compatibility' thing on
> SH3/4 hardware, I think that might be an acceptable solution too.
> Existing SH3/4 binaries are never going to run on SH2 anyway.

QEMU supports sh4 right now. If sh2 supported sh4 traps we _might_ be
able to run some sh2 code on qemu-sh4 and/or qemu-system-sh4. (But
then I dunno what the issues there are, I need to sit down and fight
with it now that elf2flt isn't blocking me.)

>> Don’t understand why we want to change personality?  More info?
>
> There's one unexpected place where the kernel has to know whether
> you're doing FDPIC or not. The sigaction syscall takes a function
> pointer for the signal handler, and on FDPIC, this is a pointer to the
> function descriptor containing the GOT pointer and actual code
> address. So if the kernel has loaded non-FDPIC ELF via the FDPIC ELF
> loader, it will have switched personality to FDPIC to treat the signal
> handler pointer specially. And then when you give it an actual
> function address instead of a function descriptor, it reads a GOT
> address and code address from the first 8 bytes of the function code,
> and blows up.
>
> If we patch the FDPIC ELF loaded to support normal ELF files (this
> should be roughly a 10-line patch) then it would never set the FDPIC
> personality for them to begin with, and no hacks to set it back would
> be needed.

Code/rodata  segment sharing is actually really _nice_ for nommu
systems. It would be nice if we could get that to work at some point.

And then there's that XIP stuff that two different ELC presentations
used for Cortex-M, the videos of which are now up at
http://elinux.org/ELC_2015_Presentations

(I refer to the talks from Jim Huang and Vitaly Wool.)

My point is, running contorted but technically valid ELF on nommu is
just a starting point, we eventually want to go beyond that to take
advantage of stuff only FDPIC can do.

>> > 5. The brk workaround I'm doing now can't be upstreamed without a
>> >   reliable runtime way to distinguish nommu. To put it in malloc.c
>> >   this would have to be a cross-arch solution. What might make more
>> >   sense is putting it in syscall_arch.h for sh
>>
>> No, this is a general nommu problem.  It will also appear on ARM,
>> ColdFire, an BlackFin, which are important targets for MUSL.
>
> Right, but I don't want to hard-code it for these archs either. In
> principle it should be possible to run an i386 binary on a nommu i386
> setup, and if the (hypothetical) kernel had a dangerour/broken brk
> there too, if also needs to be blacklisted. So what I'm looking for,
> if the kernel can't/won't just remove brk support on nommu, is a way
> to detect nommu/broken-brk and prevent it from being used. One
> simple/stupid way is:
>
> if ((size_t)&local_var - (size_t)cur_brk < MIN_DIST_TO_STACK)
>         // turn off brk support
>
> This would ban use of brk on any system where there's a risk of brk
> extending up into the stack.

Or you can check your ELF header flags and see if you've got the fdpic bit set.

>> >   where we already
>> >   have to check for SH2 to determine the right trap number; the
>> >   inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;
>>
>> I think a look at uClibc is in order.  I made it always return failure,
>> after playing with having it return results from malloc().  It’s one of
>> two things that we don’t do (or do poorly) with nommu, the other is the
>> clone() and form() family being highly restricted.
>
> It's so broken I think it should just be fixed on the kernel side.

I don't know what you mean by "fixed" here. linux/nommu.c currently has:

/*
 *  sys_brk() for the most part doesn't need the global kernel
 *  lock, except when an application is doing something nasty
 *  like trying to un-brk an area that has already been mapped
 *  to a regular file.  in this case, the unmapping will need
 *  to invoke file system routines that need the global lock.
 */
SYSCALL_DEFINE1(brk, unsigned long, brk)
{
        struct mm_struct *mm = current->mm;

        if (brk < mm->start_brk || brk > mm->context.end_brk)
                return mm->brk;

        if (mm->brk == brk)
                return mm->brk;

        /*
         * Always allow shrinking brk
         */
        if (brk <= mm->brk) {
                mm->brk = brk;
                return brk;
        }

        /*
         * Ok, looks good - let it rip.
         */
        flush_icache_range(mm->brk, brk);
        return mm->brk = brk;
}

If that should be replaced with "return -ENOSYS" we can submit a patch...

Rob


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Re: Moving forward with sh2/nommu
  2015-06-02  6:09 ` Rob Landley
@ 2015-06-02 16:45   ` Rich Felker
  2015-06-02 23:49     ` Rich Felker
  0 siblings, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-02 16:45 UTC (permalink / raw)
  To: Rob Landley; +Cc: musl

On Tue, Jun 02, 2015 at 01:09:47AM -0500, Rob Landley wrote:
> >> > 2. Kernel insists on having a stack size set in the PT_GNU_STACK
> >> >   program header; if it's 0 (the default ld produces) then execve
> >> >   fails. It should just provide a default, probably 128k (equal to
> >> >   MMU-ful Linux).
> 
> MMU-ful linux preallocates 0k and then demand faults in pages. MMU-ful
> almost never has to worry about memory fragmentation because it can
> remap _and_ move physical pages around (if nothing else, evict through
> swap).

MMU-ful commits 128k + epsilon (typically ends up being 136k), dirties
only whatever you use, and reserves a range the size of RLIMIT_STACK.
Of course on NOMMU these 3 states are not distinct; reserved,
committed, and dirty are all the same.

> >> Nooooo.  8k.  uClinux programs cannot depend on a huge stack, because that
> >> means each instance needs to kmalloc() a huge block of memory.  That is
> >> bad, but it leads to failure to load because of fragmentation (not being
> >> able to find contiguous memory blocks for all those stacks).
> >
> > My view here was just that the default, which none was specified while
> > building the program, should be something "safe". Failed execve
> > ("oops, need to use the right -Wl,-z,stack-size=XXX") is a lot easier
> > to diagnose than a stack overflow that clobbers the program code with
> > stack objects. Right now the default is "always fails to load" because
> > the kernel explicitly rejects any request for a default.
> 
> I note that Rich was probably saying he wants the default at 128k for
> ELF, not for FDPIC. That said, I'm not sure you can have a big enough
> warning sign about vanilla elf being crappy in that case.

This is unrelated to binary format, so no. It's purely a matter of
making it possible for apps to work when they're built without adding
extra CFLAGS or running extra commands to set a stack size for the
binary. My view here is that an application which was not specifically
written for NOMMU should run (or fail with a meaningful error like
ENOMEM) after compiling it with ./configure && make or equivalent
(i.e. without additional custom CFLAGS that would require
application-specific knowledge). Getting it working optimally (size,
memory usage, speed, features, etc.) in your particular environment
might require more work, of course.

Current behavior is that apps with stacksize==0 fail to run at all;
the kernel gives a mysterious error from execve (ENOEXEC?) and then
the shell tries to run the binary as a shell script. Once you
explicitly set a size, it runs with the size you asked for or fails
with ENOMEM.

Setting a small default would be much worse than the current behavior;
rather than getting errors from execve as if the binary were an
unrecognized format, you'd get massive memory corruption likely to end
with bringing down the kernel -- the stack overwrites data/code as it
expands down, then whatever got written over top of the code gets
executed.

> There's 2 things to balance here: if it doesn't "just work" then
> people are frustrated getting their programs to run, but if the
> defaults are horrible for scalability people will go "nommu is crap,
> we can't use it" without ever learning what's actually _wrong_. (It's
> a lot easier to get people to fix obvious breakage than to performance
> tune something that becomes untenable after the fact. How big the
> performance hit has to be before requiring --no-i-really-mean-it on
> the command line is an open question, but this is up there.)
> 
> Of the two ("just works" but is horrible, breaks for trivial things
> until you understand why), if they can't get "hello world" to work we
> can probably get them to read a very small HOWTO, so they at least
> know fixed stack size is an _issue_ in this context. (We're already in
> "fork does not work" territory. We are not in Kansas anymore, if you
> try to fake kansas _too_ hard you're doing developers a disservice.)

Note that for non-shared binfmts (bFLT and non-FDPIC ELF) you already
need a relatively large contiguous memory region for the program's
text+data. So I don't think a moderate-sized stack is in the realm of
"works but horrible". Rather it's on the same scale as use of
non-shareable text -- perfectly fine if you're just going to have a
few processes running after system boot, but preventing scaling. As
long as both these issues are documented (stack size and non-shared
text) I don't see it as giving a bad first impression to NOMMU users.

> That said, annotating every package in the build is silly. Probably
> there should be an enviornment variable or something that can set this
> default for entire builds, and something to actually _measure_ stack
> usage after a run would be awesome. (The kernel has checkstack.pl for
> example?) And the elf2flt command line option for setting stack size
> was just _awkward_, no idea what you've done for your fdpic binaries
> but the traditional UI for this is horrible.

There kind of is an environment variable, LDFLAGS. Using
LDFLAGS=-Wl,-z,stack-size=$SIZE will make 

> >> >   Unfortunately I suspect fixing this might be controversial since
> >> >   there may be existing binaries using brk that can't fall back to
> >> >   mmap.
> >>
> >> No, look at what I did in uClibc for brk().  I think it fails, and everything
> >> in the past depends on that.
> >
> > OK. So do you think it would be safe/acceptable to make brk always
> > fail in the kernel?
> 
> Fork does, this is _less_ intrusive than that.

Yes but fork failing isn't a 'regression' versus older kernel
versions. Personally I don't consider this a regression but a bugfix
-- brk was returning non-usable memory and doing so in such a way that
it couldn't report failure -- but I'm not sure the kernel folks will
agree.

> >> >   These numbers
> >> >   indicate "6 arguments"; there is no good reason to encode the
> >> >   number of arguments in the trap number, so we might as well just
> >> >   always use the "6 argument" code which is what the variadic
> >> >   syscall() has to use anyway. User code should then aim to use the
> >> >   correct value (22 or 38) for the model it's running on (SH3/4 or
> >> >   SH2) for compatibility with old kernels, but will still run safely
> >> >   on new kernels if it detects wrong.
> >>
> >> I say drop backward compatibility.
> >
> > That generally goes against kernel stability principles and seems
> > unlikely to be acceptable upstream.
> 
> I think Jeff means software-level backward compatibility with sh2
> wouldn't be missed. (Linux wasn't big on that architecture in the
> first place, and buildroot dropped support for it a release or two
> back.) Losing backward compatibility with sh4 would make us look
> really bad, especially since musl already supports it, but I don't
> think he's suggesting breaking sh4.
> 
> If we make sh2 binaries accept sh4 syscalls, we can call it a new
> architecture variant. (Which we actually _are_ with sh2j.) The kernel
> patch to make that work would be tiny and the hardware doesn't change.

If you mean making new sh2 binaries use the sh3/4 syscall ABI (traps
0x10-0x16 instead of 0x20-0x26) then that's a problem for being able
to support sh2a, which uses 2 of those traps for hardware exceptions.
Having division errors cause a random syscall to get invoked would not
be fun.

If on the other hand we just use only 0x16, the 6-argument form, then
it should safely avoid such clashes.

> >> > 2. Neither binutils nor gcc accepts "sh2eb-linux" as a target. Trying
> >> >   to hack it in got me a little-endian toolchain. I'm currently just
> >> >   using "sheb" and -m2 to use sh2 instructions that aren't in sh1.
> >>
> >> That is why you really want to configure —target=sh2-uclinux (or make
> >> sh2-linux do the right thing).  SH2 I think is always Big...
> >
> > Well GCC seems to consider the plain sh2-linux target as
> > little-endian.
> 
> They're crazy and broken about a lot of stuff. Sega Saturn was big
> endian. (Saturn was to sh2 what dreamcast was to sh4.)
> 
> I also note that gcc 4.2.1 and binutils 2.17 parsed sh2eb. Current
> stuff not doing so is a regression.

This should probably be reported. Are you sure it worked on unpatched
versions of the above?

> >> > 3. The complex math functions cause ICE in all gcc versions I've tried
> >> >   targetting SH2. For now we can just remove src/complex from musl,
> >> >   but that's a hack. The cause of this bug needs to be found and
> >> >   fixed in GCC.
> 
> Can I get a test program I can build and try with Aboriginal's toolchain?

I don't have a minimal test case. Here is the error:

src/complex/casin.c: In function 'casin':
src/complex/casin.c:16:1: error: unable to find a register to spill in class 'SIBCALL_REGS'
src/complex/casin.c:16:1: error: this is the insn:
...

It goes away if this file is built as non-PIC, and I never experienced
problems with sh4 with fpu, so I'm pretty sure the ICE is an
intersection of soft-float and PIC code generation. It's probably
related to compound literals too.

Adding this to musl's makefile is a temporary workaround:

$(patsubst %.c,%.o,$(wildcard src/complex/*.c)): CFLAGS += -fno-PIC -fno-PIE

> >> > musl issues:
> >> >
> >> > 1. We need runtime detection for the right trap number to use for
> >> >   syscalls. Right now I've got the trap numbers hard-coded for SH2 in
> >> >   my local tree.
> >>
> >> I don’t agree.  Just rationalise it.  Why can SH3 and above not use the
> >> same traps as SH2?
> >
> > Because the kernel syscall interface is a stable API. Even if not for
> > that, unilaterally deciding to change the interface does not instill
> > confidence in the architecture as a stable target.
> 
> And because the perception out there in linux-land is that sh4 was a
> real (if stale) processor and sh2 wasn't, so breaking sh4 to suit sh2
> _before_ we've established our new open hardware thing as actually
> viable will get the door slammed on us so hard...

Agree 100%. This can't be stressed enough.

> > OTOH if we could change to using the SH2 trap range as the default and
> > just keep the old SH3/4 range as a 'backwards compatibility' thing on
> > SH3/4 hardware, I think that might be an acceptable solution too.
> > Existing SH3/4 binaries are never going to run on SH2 anyway.
> 
> QEMU supports sh4 right now. If sh2 supported sh4 traps we _might_ be
> able to run some sh2 code on qemu-sh4 and/or qemu-system-sh4. (But
> then I dunno what the issues there are, I need to sit down and fight
> with it now that elf2flt isn't blocking me.)

qemu-sh4 supports either version of the traps. :-) This actually
helped a lot with early-stage testing of my sh2 binaries.

> >> Don’t understand why we want to change personality?  More info?
> >
> > There's one unexpected place where the kernel has to know whether
> > you're doing FDPIC or not. The sigaction syscall takes a function
> > pointer for the signal handler, and on FDPIC, this is a pointer to the
> > function descriptor containing the GOT pointer and actual code
> > address. So if the kernel has loaded non-FDPIC ELF via the FDPIC ELF
> > loader, it will have switched personality to FDPIC to treat the signal
> > handler pointer specially. And then when you give it an actual
> > function address instead of a function descriptor, it reads a GOT
> > address and code address from the first 8 bytes of the function code,
> > and blows up.
> >
> > If we patch the FDPIC ELF loaded to support normal ELF files (this
> > should be roughly a 10-line patch) then it would never set the FDPIC
> > personality for them to begin with, and no hacks to set it back would
> > be needed.
> 
> Code/rodata  segment sharing is actually really _nice_ for nommu
> systems. It would be nice if we could get that to work at some point.

Absolutely.

> And then there's that XIP stuff that two different ELC presentations
> used for Cortex-M, the videos of which are now up at
> http://elinux.org/ELC_2015_Presentations

XIP should in principle be possible with any binary that can share
code on NOMMU, but I'm not clear on whether the kernel supports it for
all of them.

> (I refer to the talks from Jim Huang and Vitaly Wool.)
> 
> My point is, running contorted but technically valid ELF on nommu is
> just a starting point, we eventually want to go beyond that to take
> advantage of stuff only FDPIC can do.

Agreed.

> >> > 5. The brk workaround I'm doing now can't be upstreamed without a
> >> >   reliable runtime way to distinguish nommu. To put it in malloc.c
> >> >   this would have to be a cross-arch solution. What might make more
> >> >   sense is putting it in syscall_arch.h for sh
> >>
> >> No, this is a general nommu problem.  It will also appear on ARM,
> >> ColdFire, an BlackFin, which are important targets for MUSL.
> >
> > Right, but I don't want to hard-code it for these archs either. In
> > principle it should be possible to run an i386 binary on a nommu i386
> > setup, and if the (hypothetical) kernel had a dangerour/broken brk
> > there too, if also needs to be blacklisted. So what I'm looking for,
> > if the kernel can't/won't just remove brk support on nommu, is a way
> > to detect nommu/broken-brk and prevent it from being used. One
> > simple/stupid way is:
> >
> > if ((size_t)&local_var - (size_t)cur_brk < MIN_DIST_TO_STACK)
> >         // turn off brk support
> >
> > This would ban use of brk on any system where there's a risk of brk
> > extending up into the stack.
> 
> Or you can check your ELF header flags and see if you've got the fdpic bit set.

Well that's a big hack (hard to do, esp in static binaries, where you
don't have the machinery to find your own headers easily from malloc),
and wouldn't help once we get binfmt_elf_fdpic to load non-FDPIC ELF
files.

I think a variant of the above check is actually what should go into
musl since it's a valid check for any system (even MMU-ful ones) and
catches a dangerous condition where brk should not be used.

> >> >   where we already
> >> >   have to check for SH2 to determine the right trap number; the
> >> >   inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;
> >>
> >> I think a look at uClibc is in order.  I made it always return failure,
> >> after playing with having it return results from malloc().  It’s one of
> >> two things that we don’t do (or do poorly) with nommu, the other is the
> >> clone() and form() family being highly restricted.
> >
> > It's so broken I think it should just be fixed on the kernel side.
> 
> I don't know what you mean by "fixed" here. linux/nommu.c currently has:

That code is not the problem. The problem is in binfmt_elf_fdpic.c:

	current->mm->brk = current->mm->start_brk;
	current->mm->context.end_brk = current->mm->start_brk;
	current->mm->context.end_brk +=
		(stack_size > PAGE_SIZE) ? (stack_size - PAGE_SIZE) : 0;
	current->mm->start_stack = current->mm->start_brk + stack_size;

It sets up a brk zone that overlaps with the stack.

In principle you could have a separate brk zone that works, but it
would require reserving a big contiguous memory range (just like for
stack) which is something you don't want on NOMMU. It's much better
for brk just to fail so mmap gets used instead.

> /*
>  *  sys_brk() for the most part doesn't need the global kernel
>  *  lock, except when an application is doing something nasty
>  *  like trying to un-brk an area that has already been mapped
>  *  to a regular file.  in this case, the unmapping will need
>  *  to invoke file system routines that need the global lock.
>  */
> SYSCALL_DEFINE1(brk, unsigned long, brk)
> {
>         struct mm_struct *mm = current->mm;
> 
>         if (brk < mm->start_brk || brk > mm->context.end_brk)
>                 return mm->brk;
> 
>         if (mm->brk == brk)
>                 return mm->brk;
> 
>         /*
>          * Always allow shrinking brk
>          */
>         if (brk <= mm->brk) {
>                 mm->brk = brk;
>                 return brk;
>         }
> 
>         /*
>          * Ok, looks good - let it rip.
>          */
>         flush_icache_range(mm->brk, brk);
>         return mm->brk = brk;
> }
> 
> If that should be replaced with "return -ENOSYS" we can submit a patch...

I'd rather have the NOMMU binfmt loaders set end_brk=start_brk, but it
wouldn't hurt to do both. At least the code that sets up the brk zone
to overlap with the stack needs to be removed. That's just an
explosion (possibly physical, depending on your application) waiting
to happen.

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Re: Moving forward with sh2/nommu
  2015-06-02 16:45   ` Rich Felker
@ 2015-06-02 23:49     ` Rich Felker
  0 siblings, 0 replies; 17+ messages in thread
From: Rich Felker @ 2015-06-02 23:49 UTC (permalink / raw)
  To: Rob Landley; +Cc: musl

On Tue, Jun 02, 2015 at 12:45:47PM -0400, Rich Felker wrote:
> > >> Nooooo.  8k.  uClinux programs cannot depend on a huge stack, because that
> > >> means each instance needs to kmalloc() a huge block of memory.  That is
> > >> bad, but it leads to failure to load because of fragmentation (not being
> > >> able to find contiguous memory blocks for all those stacks).
> > >
> > > My view here was just that the default, which none was specified while
> > > building the program, should be something "safe". Failed execve
> > > ("oops, need to use the right -Wl,-z,stack-size=XXX") is a lot easier
> > > to diagnose than a stack overflow that clobbers the program code with
> > > stack objects. Right now the default is "always fails to load" because
> > > the kernel explicitly rejects any request for a default.
> > 
> > I note that Rich was probably saying he wants the default at 128k for
> > ELF, not for FDPIC. That said, I'm not sure you can have a big enough
> > warning sign about vanilla elf being crappy in that case.
> 
> This is unrelated to binary format, so no. It's purely a matter of
> making it possible for apps to work when they're built without adding
> extra CFLAGS or running extra commands to set a stack size for the
> binary. My view here is that an application which was not specifically
> written for NOMMU should run (or fail with a meaningful error like
> ENOMEM) after compiling it with ./configure && make or equivalent
> (i.e. without additional custom CFLAGS that would require
> application-specific knowledge). Getting it working optimally (size,
> memory usage, speed, features, etc.) in your particular environment
> might require more work, of course.
> 
> Current behavior is that apps with stacksize==0 fail to run at all;
> the kernel gives a mysterious error from execve (ENOEXEC?) and then
> the shell tries to run the binary as a shell script. Once you
> explicitly set a size, it runs with the size you asked for or fails
> with ENOMEM.
> 
> Setting a small default would be much worse than the current behavior;
> rather than getting errors from execve as if the binary were an
> unrecognized format, you'd get massive memory corruption likely to end
> with bringing down the kernel -- the stack overwrites data/code as it
> expands down, then whatever got written over top of the code gets
> executed.

Slides 25-27 from the following, which came up on #musl today, are a
good reason why embedded development environments should never provide
a tiny default stack size:

http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-01 15:11 Moving forward with sh2/nommu Rich Felker
  2015-06-02  6:09 ` Rob Landley
@ 2015-06-10  3:30 ` Rich Felker
  2015-06-11  4:02   ` Rob Landley
  1 sibling, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-10  3:30 UTC (permalink / raw)
  To: musl; +Cc: rob

[-- Attachment #1: Type: text/plain, Size: 5492 bytes --]

On Mon, Jun 01, 2015 at 11:11:07AM -0400, Rich Felker wrote:
> [resent to musl list]
> 
> Here's a summary of the issues we need to work through to get a modern
> SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
> and to the point where it's something people can use roughly 'out of
> the box':
> 
> Kernel issues:
> 
> 1. Kernel should support loading plain ELF directly, unmodified. Right

I have a patch to do this, not polished but it works. It also...

> 2. Kernel insists on having a stack size set in the PT_GNU_STACK
>    program header; if it's 0 (the default ld produces) then execve
>    fails. It should just provide a default, probably 128k (equal to
>    MMU-ful Linux).

...uses a default stack size of 128k if the header is 0, and...

> 3. Kernel uses the stack for brk too, growing brk from the opposite
>    end. This is horribly buggy/dangerous. Just dummying out brk to

...set the brk size to zero so that brk will always fail.

> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
>    is that these two SH2A hardware traps overlap with the syscall
>    range used by SH3/4 ABI:

I haven't patched this yet. I'd like to use 31 (0x1f) as the new
universal SH syscall trap number, instead of 22. More details on the
reasons later.

> musl issues:
> 
> 1. We need runtime detection for the right trap number to use for
>    syscalls. Right now I've got the trap numbers hard-coded for SH2 in
>    my local tree.

I've written the runtime detection, but I'd rather not have to use it.
I managed to avoid inlining a big conditional at each syscall, but
there are still multiple ugly issues:

- The cancellable syscall code has two different instruction addresses
  where the actual syscall could take place (depending on which trap
  instruction is used), whereas the current musl C code assumes that
  there's exactly one instruction pointer range where cancellation may
  be acted upon. I have code to allow a second range but it's ugly.

- The sigreturn (sa_restorer) functions need to use runtime selection,
  which precludes putting them in the canonical form they usually
  have. At some point in the past, not using the exact instruction
  sequences expected for these functions confused gdb, but I suspect
  that's a non-issue on modern versions. Alternatively we could have
  sigreturn.c pass different versions of the sigreturn functions to
  the kernel depending on which syscall ABI is in use at runtime, but
  this requires bolting on new arch-specific logic to something that's
  normally not arch-specific, so I prefer just having the conditionals
  in the sigreturn functions.

- The syscall asm requires an extra register clobber to be able to do
  the runtime trap number switching.

An alternate design I considered was simply patching the trap number
in the .text for SH2, since there's no memory protection, but of
course this would preclude execute-in-place from ROM, etc. so I don't
think it's a proper solution. It is much less invasive, though.

> 2. We need additional runtime detection options for atomics: interrupt
>    masking for plain SH2, and the new CAS instruction for SH2J.

This is the one thing I haven't done, so currently the atomic macros
are using GUSA which is non-atomic and unsafe on SH2 (if an interrupt
happens with invalid stack pointer, memory will be corrupted). This
could be part of the random crashing I've been experiencing (although
I reproduced it without musl) so I'll try to add them next.

> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
>    won't work. I have a version locally but it doesn't make sense to
>    commit without runtime trap number selection.

Done and updated to use runtime selection in the (ugly) patch.

> 4. As long as we're using the FDPIC ELF header flag to get
>    binfmt_elf_fdpic.c to load binaries, the startup code needs to call
>    the personality() syscall to switch back. I have a local hack for
>    doing this in rcrt1.o which is probably not worth upstreaming if we
>    can just make the kernel do it right.

No longer needed because of the kernel patch to load normal ELF.

> 5. The brk workaround I'm doing now can't be upstreamed without a
>    reliable runtime way to distinguish nommu. To put it in malloc.c
>    this would have to be a cross-arch solution. What might make more
>    sense is putting it in syscall_arch.h for sh, where we already
>    have to check for SH2 to determine the right trap number; the
>    inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;

Commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 in musl solves this in
a general manner, even though it's no longer needed with my kernel
patch applied.

One more musl-side issue I neglected to mention is the __unmapself.s
can't work on SH2 because the SH2 trap/interrupt mechanism requires
the userspace stack pointer to be valid at all times. This is now
solved upstream in commit c30cbcb0a646b1f13a22c645616dce624465b883,
but activating it for SH2 requires removing
src/thread/sh/__unmapself.s so the generic C file gets used.

The attached patch covers everything described above that's not
already upstream, and is sufficient to build musl for sh2 with
musl-cross targeting "sheb-linux-musl". I used gcc 4.7.3 because later
versions break the kernel. The attached config.mak for musl shows the
configure options I used. The attached sheb.specs file is how I got
gcc to do always-PIE without breaking the kernel.

Rich

[-- Attachment #2: fdpic_elf_loader.diff --]
[-- Type: text/plain, Size: 1969 bytes --]

--- fs/binfmt_elf_fdpic.c.orig
+++ fs/binfmt_elf_fdpic.c
@@ -103,14 +103,27 @@
 core_initcall(init_elf_fdpic_binfmt);
 module_exit(exit_elf_fdpic_binfmt);
 
+static int is_fdpic(struct elfhdr *hdr)
+{
+#ifdef CONFIG_MMU
+	return 1;
+#else
+	return elf_check_fdpic(hdr);
+#endif
+}
+
 static int is_elf_fdpic(struct elfhdr *hdr, struct file *file)
 {
 	if (memcmp(hdr->e_ident, ELFMAG, SELFMAG) != 0)
 		return 0;
 	if (hdr->e_type != ET_EXEC && hdr->e_type != ET_DYN)
 		return 0;
-	if (!elf_check_arch(hdr) || !elf_check_fdpic(hdr))
+	if (!elf_check_arch(hdr))
 		return 0;
+#ifdef CONFIG_MMU
+	if (!elf_check_fdpic(hdr))
+		return 0;
+#endif
 	if (!file->f_op->mmap)
 		return 0;
 	return 1;
@@ -269,7 +282,7 @@
 
 	}
 
-	if (elf_check_const_displacement(&exec_params.hdr))
+	if (elf_check_const_displacement(&exec_params.hdr) || !is_fdpic(&exec_params.hdr))
 		exec_params.flags |= ELF_FDPIC_FLAG_CONSTDISP;
 
 	/* perform insanity checks on the interpreter */
@@ -306,9 +319,9 @@
 
 	retval = -ENOEXEC;
 	if (stack_size == 0)
-		goto error;
+		stack_size = 131072;
 
-	if (elf_check_const_displacement(&interp_params.hdr))
+	if (elf_check_const_displacement(&interp_params.hdr) || !is_fdpic(&interp_params.hdr))
 		interp_params.flags |= ELF_FDPIC_FLAG_CONSTDISP;
 
 	/* flush all traces of the currently running executable */
@@ -319,7 +332,8 @@
 	/* there's now no turning back... the old userspace image is dead,
 	 * defunct, deceased, etc.
 	 */
-	set_personality(PER_LINUX_FDPIC);
+	if (is_fdpic(&exec_params.hdr))
+		set_personality(PER_LINUX_FDPIC);
 	if (elf_read_implies_exec(&exec_params.hdr, executable_stack))
 		current->personality |= READ_IMPLIES_EXEC;
 
@@ -400,8 +414,6 @@
 
 	current->mm->brk = current->mm->start_brk;
 	current->mm->context.end_brk = current->mm->start_brk;
-	current->mm->context.end_brk +=
-		(stack_size > PAGE_SIZE) ? (stack_size - PAGE_SIZE) : 0;
 	current->mm->start_stack = current->mm->start_brk + stack_size;
 #endif
 

[-- Attachment #3: musl_sh2_v2.diff --]
[-- Type: text/plain, Size: 13622 bytes --]

diff --git a/Makefile b/Makefile
index 2eb7b30..51baeab 100644
--- a/Makefile
+++ b/Makefile
@@ -108,6 +108,8 @@ $(NOSSP_SRCS:%.c=%.o) $(NOSSP_SRCS:%.c=%.lo): CFLAGS += $(CFLAGS_NOSSP)
 
 $(CRT_LIBS:lib/%=crt/%): CFLAGS += -DCRT
 
+$(patsubst %.c,%.o,$(wildcard src/complex/*.c)): CFLAGS += -fno-PIC -fno-PIE
+
 # This incantation ensures that changes to any subarch asm files will
 # force the corresponding object file to be rebuilt, even if the implicit
 # rule below goes indirectly through a .sub file.
diff --git a/arch/sh/pthread_arch.h b/arch/sh/pthread_arch.h
index 65c389f..0960422 100644
--- a/arch/sh/pthread_arch.h
+++ b/arch/sh/pthread_arch.h
@@ -9,3 +9,14 @@ static inline struct pthread *__pthread_self()
 #define TP_ADJ(p) ((char *)(p) + sizeof(struct pthread) - 8)
 
 #define CANCEL_REG_IP 17
+
+#ifdef SHARED
+__attribute__((__visibility__("hidden")))
+#endif
+extern const char __cp_begin[1], __cp_end[1],
+	__cp2_begin[1], __cp2_end[1];
+
+#define IN_CP(ip) ((uintptr_t)(ip)-(uintptr_t)__cp_begin \
+	< (uintptr_t)__cp_end-(uintptr_t)__cp_begin \
+	|| (uintptr_t)(ip)-(uintptr_t)__cp2_begin \
+	< (uintptr_t)__cp2_end-(uintptr_t)__cp2_begin)
diff --git a/arch/sh/src/__set_thread_area.c b/arch/sh/src/__set_thread_area.c
index e69de29..a723ff2 100644
--- a/arch/sh/src/__set_thread_area.c
+++ b/arch/sh/src/__set_thread_area.c
@@ -0,0 +1,21 @@
+#include "pthread_impl.h"
+#include "libc.h"
+#include <elf.h>
+
+/* Also perform sh-specific init */
+
+__attribute__((__visibility__("hidden"))) int __sh2_abi;
+
+int __set_thread_area(void *p)
+{
+	size_t *aux;
+	__asm__ __volatile__ ( "ldc %0, gbr" : : "r"(p) : "memory" );
+	for (aux=libc.auxv; *aux; aux+=2) {
+		if (*aux != AT_PLATFORM) continue;
+		const char *s = (void *)aux[1];
+		if (s[0]!='s' || s[1]!='h' || s[2]!='2' || s[3]-'0'<10u) break;
+		__sh2_abi = 1;
+		break;
+	}
+	return 0;
+}
diff --git a/arch/sh/src/atomic.c b/arch/sh/src/atomic.c
index f8c615f..b3a8968 100644
--- a/arch/sh/src/atomic.c
+++ b/arch/sh/src/atomic.c
@@ -27,6 +27,8 @@
 
 #define CPU_HAS_LLSC 0x0040
 
+#define __hwcap 0
+
 int __sh_cas(volatile int *p, int t, int s)
 {
 	if (__hwcap & CPU_HAS_LLSC) return __sh_cas_llsc(p, t, s);
diff --git a/arch/sh/src/sh/sh_syscall.s b/arch/sh/src/sh/sh_syscall.s
index e69de29..546ae07 100644
--- a/arch/sh/src/sh/sh_syscall.s
+++ b/arch/sh/src/sh/sh_syscall.s
@@ -0,0 +1,38 @@
+.hidden __sh_syscall
+.global __sh_syscall
+.type __sh_syscall,@function
+__sh_syscall:
+	mov.l r0,@-r15
+	mov.l r2,@-r15
+
+	mova  1f, r0
+	mov.l 1f, r2
+	add   r0, r2
+	mov.l @r2, r2
+	tst   r2, r2
+
+	mov.l @r15+, r2
+	mov.l @r15+, r0
+
+	bt    3f
+	bra   2f
+	 nop
+	.align 2
+	.hidden __sh2_abi
+1:	.long __sh2_abi@PCREL
+
+2:	trapa #38
+	bra 1f
+	 nop
+
+3:	trapa #22
+	or    r0, r0
+	or    r0, r0
+	or    r0, r0
+	or    r0, r0
+	or    r0, r0
+
+1:	neg r2, r2
+	add #__sh_syscall-9f, r2
+9:	braf r2
+	 nop
diff --git a/arch/sh/syscall_arch.h b/arch/sh/syscall_arch.h
index 7ee21a5..b56dfbf 100644
--- a/arch/sh/syscall_arch.h
+++ b/arch/sh/syscall_arch.h
@@ -3,26 +3,25 @@
 ((union { long long ll; long l[2]; }){ .ll = x }).l[1]
 #define __SYSCALL_LL_O(x) __SYSCALL_LL_E((x))
 
-/* The extra OR instructions are to work around a hardware bug:
- * http://documentation.renesas.com/doc/products/mpumcu/tu/tnsh7456ae.pdf
- */
-#define __asm_syscall(trapno, ...) do {   \
-	__asm__ __volatile__ (                \
-		"trapa #" #trapno "\n"            \
-		"or r0, r0\n"                     \
-		"or r0, r0\n"                     \
-		"or r0, r0\n"                     \
-		"or r0, r0\n"                     \
-		"or r0, r0\n"                     \
-	: "=r"(r0) : __VA_ARGS__ : "memory"); \
-	return r0;                            \
-	} while (0)
+#define __SYSCALL_ASM "\n"           \
+"	mov.l 1f, r2\n"              \
+"	.align 2\n"                  \
+"2:	braf r2\n"                   \
+"	 nop\n"                      \
+"	.hidden __sh_syscall\n"      \
+"1:	.long __sh_syscall@PCREL\n"  \
+"3:	\n"
+
+#define __asm_syscall(...) \
+	__asm__ __volatile__ ( __SYSCALL_ASM \
+	: "=r"(r0) : __VA_ARGS__ : "memory", "r2", "t" )
 
 static inline long __syscall0(long n)
 {
 	register long r3 __asm__("r3") = n;
 	register long r0 __asm__("r0");
-	__asm_syscall(16, "r"(r3));
+	__asm_syscall("r"(r3));
+	return r0;
 }
 
 static inline long __syscall1(long n, long a)
@@ -30,7 +29,8 @@ static inline long __syscall1(long n, long a)
 	register long r3 __asm__("r3") = n;
 	register long r4 __asm__("r4") = a;
 	register long r0 __asm__("r0");
-	__asm_syscall(17, "r"(r3), "r"(r4));
+	__asm_syscall("r"(r3), "r"(r4));
+	return r0;
 }
 
 static inline long __syscall2(long n, long a, long b)
@@ -39,7 +39,8 @@ static inline long __syscall2(long n, long a, long b)
 	register long r4 __asm__("r4") = a;
 	register long r5 __asm__("r5") = b;
 	register long r0 __asm__("r0");
-	__asm_syscall(18, "r"(r3), "r"(r4), "r"(r5));
+	__asm_syscall("r"(r3), "r"(r4), "r"(r5));
+	return r0;
 }
 
 static inline long __syscall3(long n, long a, long b, long c)
@@ -49,7 +50,8 @@ static inline long __syscall3(long n, long a, long b, long c)
 	register long r5 __asm__("r5") = b;
 	register long r6 __asm__("r6") = c;
 	register long r0 __asm__("r0");
-	__asm_syscall(19, "r"(r3), "r"(r4), "r"(r5), "r"(r6));
+	__asm_syscall("r"(r3), "r"(r4), "r"(r5), "r"(r6));
+	return r0;
 }
 
 static inline long __syscall4(long n, long a, long b, long c, long d)
@@ -60,7 +62,8 @@ static inline long __syscall4(long n, long a, long b, long c, long d)
 	register long r6 __asm__("r6") = c;
 	register long r7 __asm__("r7") = d;
 	register long r0 __asm__("r0");
-	__asm_syscall(20, "r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7));
+	__asm_syscall("r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7));
+	return r0;
 }
 
 static inline long __syscall5(long n, long a, long b, long c, long d, long e)
@@ -71,7 +74,8 @@ static inline long __syscall5(long n, long a, long b, long c, long d, long e)
 	register long r6 __asm__("r6") = c;
 	register long r7 __asm__("r7") = d;
 	register long r0 __asm__("r0") = e;
-	__asm_syscall(21, "r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7), "0"(r0));
+	__asm_syscall("r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7), "0"(r0));
+	return r0;
 }
 
 static inline long __syscall6(long n, long a, long b, long c, long d, long e, long f)
@@ -83,5 +87,6 @@ static inline long __syscall6(long n, long a, long b, long c, long d, long e, lo
 	register long r7 __asm__("r7") = d;
 	register long r0 __asm__("r0") = e;
 	register long r1 __asm__("r1") = f;
-	__asm_syscall(22, "r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7), "0"(r0), "r"(r1));
+	__asm_syscall("r"(r3), "r"(r4), "r"(r5), "r"(r6), "r"(r7), "0"(r0), "r"(r1));
+	return r0;
 }
diff --git a/src/env/__libc_start_main.c b/src/env/__libc_start_main.c
index f6f3b14..0ab80e2 100644
--- a/src/env/__libc_start_main.c
+++ b/src/env/__libc_start_main.c
@@ -42,6 +42,8 @@ void __init_libc(char **envp, char *pn)
 	__init_tls(aux);
 	__init_ssp((void *)aux[AT_RANDOM]);
 
+	if (__syscall(SYS_personality, 0) == -1) a_crash();
+
 	if (aux[AT_UID]==aux[AT_EUID] && aux[AT_GID]==aux[AT_EGID]
 		&& !aux[AT_SECURE]) return;
 
diff --git a/src/internal/sh/syscall.s b/src/internal/sh/syscall.s
index d00712a..23feb32 100644
--- a/src/internal/sh/syscall.s
+++ b/src/internal/sh/syscall.s
@@ -2,10 +2,6 @@
 .hidden __syscall
 .type   __syscall, @function
 __syscall:
-	! The kernel syscall entry point documents that the trap number indicates
-	! the number of arguments being passed, but it then ignores that information.
-	! Since we do not actually know how many arguments are being passed, we will
-	! say there are six, since that is the maximum we support here.
 	mov r4, r3
 	mov r5, r4
 	mov r6, r5
@@ -13,11 +9,13 @@ __syscall:
 	mov.l @r15, r7
 	mov.l @(4,r15), r0
 	mov.l @(8,r15), r1
-	trapa #22
-	or r0, r0
-	or r0, r0
-	or r0, r0
-	or r0, r0
-	or r0, r0
+
+	mov.l 1f, r2
+	.align 2
+2:	braf r2
+	 nop
+	.hidden __sh_syscall
+1:	.long __sh_syscall@PCREL
+
 	rts
 	 nop
diff --git a/src/process/sh/vfork.s b/src/process/sh/vfork.s
index e69de29..9c61d03 100644
--- a/src/process/sh/vfork.s
+++ b/src/process/sh/vfork.s
@@ -0,0 +1,23 @@
+.global __vfork
+.weak vfork
+.type __vfork,@function
+.type vfork,@function
+__vfork:
+vfork:
+	mov #95, r3
+	add r3, r3
+
+	mov.l 1f, r2
+	.align 2
+2:	braf r2
+	 nop
+	.hidden __sh_syscall
+1:	.long __sh_syscall@PCREL
+
+	mov r0, r4
+	mov.l 1f, r0
+2:	braf r0
+	 nop
+	.align 2
+	.hidden __syscall_ret
+1:	.long __syscall_ret@PLT-(2b+4-.)
diff --git a/src/signal/sh/restore.s b/src/signal/sh/restore.s
index ab26034..f1b462c 100644
--- a/src/signal/sh/restore.s
+++ b/src/signal/sh/restore.s
@@ -2,23 +2,18 @@
 .type   __restore, @function
 __restore:
 	mov   #119, r3  !__NR_sigreturn
-	trapa #16
-
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
+	bra 1f
+	 nop
 
 .global __restore_rt
 .type   __restore_rt, @function
 __restore_rt:
 	mov   #100, r3  !__NR_rt_sigreturn
 	add   #73, r3
-	trapa #16
 
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
+1:	mov.l 1f, r2
+	.align 2
+2:	braf r2
+	 nop
+	.hidden __sh_syscall
+1:	.long __sh_syscall@PCREL
diff --git a/src/thread/pthread_cancel.c b/src/thread/pthread_cancel.c
index 0151a1a..5ddd304 100644
--- a/src/thread/pthread_cancel.c
+++ b/src/thread/pthread_cancel.c
@@ -57,6 +57,11 @@ __attribute__((__visibility__("hidden")))
 #endif
 extern const char __cp_begin[1], __cp_end[1];
 
+#ifndef IN_CP
+#define IN_CP(ip) ((uintptr_t)(ip)-(uintptr_t)__cp_begin \
+	< (uintptr_t)__cp_end-(uintptr_t)__cp_begin)
+#endif
+
 static void cancel_handler(int sig, siginfo_t *si, void *ctx)
 {
 	pthread_t self = __pthread_self();
@@ -68,7 +73,7 @@ static void cancel_handler(int sig, siginfo_t *si, void *ctx)
 
 	_sigaddset(&uc->uc_sigmask, SIGCANCEL);
 
-	if (self->cancelasync || ip >= __cp_begin && ip < __cp_end) {
+	if (self->cancelasync || IN_CP(ip)) {
 		((char **)&uc->uc_mcontext)[CANCEL_REG_IP] = (char *)__cp_cancel;
 		return;
 	}
diff --git a/src/thread/sh/__set_thread_area.s b/src/thread/sh/__set_thread_area.s
index d9f1181..e69de29 100644
--- a/src/thread/sh/__set_thread_area.s
+++ b/src/thread/sh/__set_thread_area.s
@@ -1,6 +0,0 @@
-.global __set_thread_area
-.type   __set_thread_area, @function
-__set_thread_area:
-	ldc r4, gbr
-	rts
-	 mov #0, r0
diff --git a/src/thread/sh/__unmapself.s b/src/thread/sh/__unmapself.s
deleted file mode 100644
index b34c3c8..0000000
--- a/src/thread/sh/__unmapself.s
+++ /dev/null
@@ -1,22 +0,0 @@
-.text
-.global __unmapself
-.type   __unmapself, @function
-__unmapself:
-	mov   #91, r3  ! SYS_munmap
-	trapa #18
-
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-
-	mov   #1, r3   ! SYS_exit
-	mov   #0, r4
-	trapa #17
-
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
-	or    r0, r0
diff --git a/src/thread/sh/clone.s b/src/thread/sh/clone.s
index d6c9184..5fc97fc 100644
--- a/src/thread/sh/clone.s
+++ b/src/thread/sh/clone.s
@@ -9,7 +9,7 @@ __clone:
 	and   r0, r5
 
 	mov   r4, r1         ! r1 = fn
-	mov   r7, r2         ! r2 = arg
+	mov.l r7, @-r5       ! (r5) = arg
 
 	mov   #120,     r3   ! r3 = __NR_clone
 	mov   r6,       r4   ! r4 = flags
@@ -17,13 +17,13 @@ __clone:
 	mov.l @r15,     r6   ! r6 = ptid
 	mov.l @(8,r15), r7   ! r7 = ctid
 	mov.l @(4,r15), r0   ! r0 = tls
-	trapa #21
 
-	or r0, r0
-	or r0, r0
-	or r0, r0
-	or r0, r0
-	or r0, r0
+3:	mov.l 1f, r2
+	.align 2
+2:	braf r2
+	 nop
+	.hidden __sh_syscall
+1:	.long __sh_syscall@PCREL
 
 	cmp/eq #0, r0
 	bt     1f
@@ -34,14 +34,9 @@ __clone:
 
 1:	! we are the child, call fn(arg)
 	jsr    @r1
-	 mov   r2, r4
+	 mov.l @r15+, r4
 
 	mov   #1, r3   ! __NR_exit
 	mov   r0, r4
-	trapa #17
-
-	or   r0, r0
-	or   r0, r0
-	or   r0, r0
-	or   r0, r0
-	or   r0, r0
+	bra   3b
+	 nop
diff --git a/src/thread/sh/syscall_cp.s b/src/thread/sh/syscall_cp.s
index 6b28ddf..f8bb607 100644
--- a/src/thread/sh/syscall_cp.s
+++ b/src/thread/sh/syscall_cp.s
@@ -1,4 +1,8 @@
 .text
+.global __cp2_begin
+.hidden __cp2_begin
+.global __cp2_end
+.hidden __cp2_end
 .global __cp_begin
 .hidden __cp_begin
 .global __cp_end
@@ -11,28 +15,58 @@
 .type   __syscall_cp_asm, @function
 __syscall_cp_asm:
 
-__cp_begin:
-	mov.l @r4, r4
-	tst   r4, r4
-	bt    2f
-
-	mov.l L1, r0
-	braf  r0
-	 nop
-1:
-
-.align 2
-L1:	.long __cancel@PLT-(1b-.)
+	mova  1f, r0
+	mov.l 1f, r2
+	add   r0, r2
+	mov.l @r1, r2
+	tst   r2, r2
 
-2:	mov   r5, r3
+	! setup syscall args
+	mov   r4, r2
+	mov   r5, r3
 	mov   r6, r4
 	mov   r7, r5
 	mov.l @r15, r6
 	mov.l @(4,r15), r7
 	mov.l @(8,r15), r0
 	mov.l @(12,r15), r1
-	trapa #22
 
+	bt    __cp_begin
+	bra   __cp2_begin
+	 nop
+	.align 2
+	.hidden __sh2_abi
+1:	.long __sh2_abi@PCREL
+
+2:
+__cp2_begin:
+	mov.l @r2, r2
+	tst   r2, r2
+	bt    2f
+
+	mov.l 1f, r0
+	.align 2
+	braf  r0
+	 nop
+1:	.long __cancel@PCREL
+
+2:	trapa #38
+__cp2_end:
+	rts
+	 nop
+
+__cp_begin:
+	mov.l @r2, r2
+	tst   r2, r2
+	bt    2f
+
+	mov.l 1f, r0
+	.align 2
+	braf  r0
+	 nop
+1:	.long __cancel@PCREL
+
+2:	trapa #22
 __cp_end:
 	! work around hardware bug
 	or   r0, r0
@@ -40,6 +74,5 @@ __cp_end:
 	or   r0, r0
 	or   r0, r0
 	or   r0, r0
-
 	rts
 	 nop
diff --git a/src/unistd/sh/pipe.s b/src/unistd/sh/pipe.s
index d865ae3..ba3ea10 100644
--- a/src/unistd/sh/pipe.s
+++ b/src/unistd/sh/pipe.s
@@ -2,14 +2,13 @@
 .type   pipe, @function
 pipe:
 	mov    #42, r3
-	trapa  #17
 
-	! work around hardware bug
-	or     r0, r0
-	or     r0, r0
-	or     r0, r0
-	or     r0, r0
-	or     r0, r0
+	mov.l 1f, r2
+	.align 2
+2:	braf r2
+	 nop
+	.hidden __sh_syscall
+1:	.long __sh_syscall@PCREL
 
 	cmp/pz r0
 	bt     1f

[-- Attachment #4: config.mak --]
[-- Type: text/plain, Size: 1039 bytes --]

# This version of config.mak was generated by:
# ./configure --prefix=/opt/sheb-linux-musl/sheb-linux-musl CC=sheb-linux-musl-gcc CFLAGS='-fPIC -m2' --disable-shared
# Any changes made here will be lost if configure is re-run
ARCH = sh
SUBARCH = eb-nofpu
ASMSUBARCH = eb-nofpu
prefix = /opt/sheb-linux-musl/sheb-linux-musl
exec_prefix = $(prefix)
bindir = $(exec_prefix)/bin
libdir = $(prefix)/lib
includedir = $(prefix)/include
syslibdir = /lib
CC = sheb-linux-musl-gcc
CFLAGS = -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fPIC -m2
CFLAGS_C99FSE = -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math
CFLAGS_MEMOPS = -fno-tree-loop-distribute-patterns
CFLAGS_NOSSP = -fno-stack-protector
CPPFLAGS = 
LDFLAGS = -Wl,--hash-style=both 
CROSS_COMPILE = 
LIBCC = -lgcc
OPTIMIZE_GLOBS = internal/*.c malloc/*.c string/*.c
SHARED_LIBS =
ALL_TOOLS =
TOOL_LIBS =

[-- Attachment #5: specs.sheb --]
[-- Type: text/plain, Size: 5188 bytes --]

*asm:
%(subtarget_asm_endian_spec) %{mrelax:-relax %(subtarget_asm_relax_spec)}%(subtarget_asm_isa_spec) %(subtarget_asm_spec)%{m2a:--isa=sh2a} %{m2a-single:--isa=sh2a} %{m2a-single-only:--isa=sh2a} %{m2a-nofpu:--isa=sh2a-nofpu} %{m5-compact*:--isa=SHcompact} %{m5-32media*:--isa=SHmedia --abi=32} %{m5-64media*:--isa=SHmedia --abi=64} %{m4al:-dsp} %{mcut2-workaround:-cut2-workaround}

*asm_debug:


*asm_final:


*asm_options:
%{-target-help:%:print-asm-header()} %{v} %{w:-W} %{I*} %a %Y %{c:%W{o*}%{!o*:-o %w%b%O}}%{!c:-o %d%w%u%O}

*invoke_as:
%{!fwpa:   %{fcompare-debug=*|fdump-final-insns=*:%:compare-debug-dump-opt()}   %{!S:-o %|.s |
 as %(asm_options) %m.s %A }  }

*cpp:
 %(subtarget_cpp_spec) 

*cpp_options:
%(cpp_unique_options) %1 %{m*} %{std*&ansi&trigraphs} %{W*&pedantic*} %{w} %{f*} %{g*:%{!g0:%{g*} %{!fno-working-directory:-fworking-directory}}} %{O*} %{undef} %{save-temps*:-fpch-preprocess}

*cpp_debug_options:
%{d*}

*cpp_unique_options:
%{!Q:-quiet} %{nostdinc*} %{C} %{CC} %{v} %{I*&F*} %{P} %I %{MD:-MD %{!o:%b.d}%{o*:%.d%*}} %{MMD:-MMD %{!o:%b.d}%{o*:%.d%*}} %{M} %{MM} %{MF*} %{MG} %{MP} %{MQ*} %{MT*} %{!E:%{!M:%{!MM:%{!MT:%{!MQ:%{MD|MMD:%{o*:-MQ %*}}}}}}} %{remap} %{g3|ggdb3|gstabs3|gcoff3|gxcoff3|gvms3:-dD} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %{H} %C %{D*&U*&A*} %{i*} %Z %i %{fmudflap:-D_MUDFLAP -include mf-runtime.h} %{fmudflapth:-D_MUDFLAP -D_MUDFLAPTH -include mf-runtime.h} %{E|M|MM:%W{o*}}

*trad_capable_cpp:
cc1 -E %{traditional|traditional-cpp:-traditional-cpp}

*cc1:
%{profile:-p}

*cc1_options:
%{D__KERNEL__:;:-fPIE} %{pg:%{fomit-frame-pointer:%e-pg and -fomit-frame-pointer are incompatible}} %{!iplugindir*:%{fplugin*:%:find-plugindir()}} %1 %{!Q:-quiet} %{!dumpbase:-dumpbase %B} %{d*} %{m*} %{aux-info*} %{fcompare-debug-second:%:compare-debug-auxbase-opt(%b)}  %{!fcompare-debug-second:%{c|S:%{o*:-auxbase-strip %*}%{!o*:-auxbase %b}}}%{!c:%{!S:-auxbase %b}}  %{g*} %{O*} %{W*&pedantic*} %{w} %{std*&ansi&trigraphs} %{v:-version} %{pg:-p} %{p} %{f*} %{undef} %{Qn:-fno-ident} %{Qy:} %{-help:--help} %{-target-help:--target-help} %{-version:--version} %{-help=*:--help=%*} %{!fsyntax-only:%{S:%W{o*}%{!o*:-o %b.s}}} %{fsyntax-only:-o %j} %{-param*} %{fmudflap|fmudflapth:-fno-builtin -fno-merge-constants} %{coverage:-fprofile-arcs -ftest-coverage}

*cc1plus:


*link_gcc_c_sequence:
%{static:--start-group} %G %L %{static:--end-group}%{!static:%G}

*link_ssp:
%{fstack-protector:}

*endfile:
crtendS.o%s crtn.o%s

*link:
%{!static:--eh-frame-hdr} -m %(link_emul_prefix)%{m5-compact*|m5-32media*:32}%{m5-64media*:64}%{!m1:%{!m2:%{!m3*:%{!m4*:%{!m5*:%(link_default_cpu_emul)}}}}}%(subtarget_link_emul_suffix) %{mrelax:-relax} %(subtarget_link_spec)

*lib:
%{pthread:-lpthread}    %{shared:-lc}    %{!shared:%{mieee-fp:-lieee} %{profile:-lc_p}%{!profile:-lc}}

*mfwrap:
 %{static: %{fmudflap|fmudflapth:  --wrap=malloc --wrap=free --wrap=calloc --wrap=realloc --wrap=mmap --wrap=mmap64 --wrap=munmap --wrap=alloca} %{fmudflapth: --wrap=pthread_create}} %{fmudflap|fmudflapth: --wrap=main}

*mflib:
%{fmudflap|fmudflapth: -export-dynamic}

*link_gomp:


*libgcc:
-lgcc -lgcc_eh

*startfile:
%{!shared:rcrt1.o%s}    crti.o%s crtbeginS.o%s

*cross_compile:
1

*version:
4.7.3

*multilib:
. ;

*multilib_defaults:
mb m1

*multilib_extra:


*multilib_matches:


*multilib_exclusions:


*multilib_options:


*linker:
collect2

*linker_plugin_file:


*lto_wrapper:


*lto_gcc:


*link_libgcc:
%D

*md_exec_prefix:


*md_startfile_prefix:


*md_startfile_prefix_1:


*startfile_prefix_spec:


*sysroot_spec:
--sysroot=%R

*sysroot_suffix_spec:


*sysroot_hdrs_suffix_spec:


*self_spec:


*subtarget_cpp_spec:
   %{posix:-D_POSIX_SOURCE}    %{pthread:-D_REENTRANT -D_PTHREADS} 

*link_emul_prefix:
sh%{ml:l}elf

*link_default_cpu_emul:


*subtarget_link_emul_suffix:
_linux

*subtarget_link_spec:
%{shared:-shared}    %{!static:      %{rdynamic:-export-dynamic}      -dynamic-linker %{mglibc:/lib/ld-linux.so.2;:%{muclibc:/lib/ld-uClibc.so.0;:%{mbionic:/system/bin/linker;:/lib/ld-musl-sheb-nofpu.so.1}}}}    %{static:-static}

*subtarget_asm_endian_spec:
%{ml:-little} %{!ml:-big}

*subtarget_asm_relax_spec:
%{m4*:-isa=sh4-up}

*subtarget_asm_isa_spec:


*subtarget_asm_spec:


*link_command:
%{!fsyntax-only:%{!c:%{!M:%{!MM:%{!E:%{!S:    %(linker) %{!fno-use-linker-plugin:%{flto|flto=*|fuse-linker-plugin:     -plugin %(linker_plugin_file)     -plugin-opt=%(lto_wrapper)     -plugin-opt=-fresolution=%u.res     %{!nostdlib:%{!nodefaultlibs:%:pass-through-libs(%(link_gcc_c_sequence))}}     }}%{flto|flto=*:%<fcompare-debug*}     %{flto} %{flto=*} %l %{pie:} %{!nostdlib:%{!r:%{!shared:-shared -Bsymbolic -Bstatic -z stack-size=131072}}} %X %{o*} %{e*} %{N} %{n} %{r}    %{s} %{t} %{u*} %{z} %{Z} %{!nostdlib:%{!nostartfiles:%S}}    %{static:} %{L*} %(mfwrap) %(link_libgcc) %o    %{fopenmp|ftree-parallelize-loops=*:%:include(libgomp.spec)%(link_gomp)}    %{fgnu-tm:%:include(libitm.spec)%(link_itm)}    %(mflib)  %{fsplit-stack: --wrap=pthread_create}    %{fprofile-arcs|fprofile-generate*|
 coverage:-lgcov}    %{!nostdlib:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}}    %{!nostdlib:%{!nostartfiles:%E}} %{T*} }}}}}}


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-10  3:30 ` Rich Felker
@ 2015-06-11  4:02   ` Rob Landley
  2015-06-11 15:12     ` Rich Felker
  0 siblings, 1 reply; 17+ messages in thread
From: Rob Landley @ 2015-06-11  4:02 UTC (permalink / raw)
  To: Rich Felker, musl, D. Jeff Dionne, ysato, shumpei.kawasaki



On 06/09/2015 10:30 PM, Rich Felker wrote:
> On Mon, Jun 01, 2015 at 11:11:07AM -0400, Rich Felker wrote:
>> [resent to musl list]
>>
>> Here's a summary of the issues we need to work through to get a modern
>> SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
>> and to the point where it's something people can use roughly 'out of
>> the box':
>>
>> Kernel issues:
>>
>> 1. Kernel should support loading plain ELF directly, unmodified. Right
> 
> I have a patch to do this, not polished but it works. It also...
> 
>> 2. Kernel insists on having a stack size set in the PT_GNU_STACK
>>    program header; if it's 0 (the default ld produces) then execve
>>    fails. It should just provide a default, probably 128k (equal to
>>    MMU-ful Linux).
> 
> ...uses a default stack size of 128k if the header is 0, and...

That's big enough to give a false sense of security without being big
enough to actually work reliably on software that isn't thinking about it.

The kernel stack size is apparently 4k, but they've got more than one
for different contexts:

https://www.kernel.org/doc/Documentation/x86/x86_64/kernel-stacks

I think I agree to jeff here: the historical default stack size for
nommu packaging was 8k. That's enough for hello world to work, but in
this context stack is a resource we need to allocate if we want
nontrivial amounts of it.

>> 3. Kernel uses the stack for brk too, growing brk from the opposite
>>    end. This is horribly buggy/dangerous. Just dummying out brk to
> 
> ...set the brk size to zero so that brk will always fail.

Yay always failing. The brk approach does not make sense in this context.

>> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
>>    is that these two SH2A hardware traps overlap with the syscall
>>    range used by SH3/4 ABI:
> 
> I haven't patched this yet. I'd like to use 31 (0x1f) as the new
> universal SH syscall trap number, instead of 22. More details on the
> reasons later.

I've cc'd Yoshinori Sato (who did most of the historical sh2 work) and
Shumpei Kawasaki (the original superh architect). They'll probably have
an opinion on your "more reasons" for changing sh2 system call numbers
to match sh4.

>> musl issues:
>>
>> 1. We need runtime detection for the right trap number to use for
>>    syscalls. Right now I've got the trap numbers hard-coded for SH2 in
>>    my local tree.
> 
> I've written the runtime detection, but I'd rather not have to use it.
> I managed to avoid inlining a big conditional at each syscall, but
> there are still multiple ugly issues:

This is only an issue if you want sh4 to be able to run sh2 binaries.
(Can m68k run coldfire binaries? Last I checked there wasn't any
blackfin-with-mmu variant.)

Rebuilding your source for a different output format really isn't a big
deal in open source. I know you wanted musl binary compatible with glibc
too, but I don't see the advantage.

> - The cancellable syscall code has two different instruction addresses
>   where the actual syscall could take place (depending on which trap
>   instruction is used), whereas the current musl C code assumes that
>   there's exactly one instruction pointer range where cancellation may
>   be acted upon. I have code to allow a second range but it's ugly.
> 
> - The sigreturn (sa_restorer) functions need to use runtime selection,
>   which precludes putting them in the canonical form they usually
>   have. At some point in the past, not using the exact instruction
>   sequences expected for these functions confused gdb, but I suspect
>   that's a non-issue on modern versions. Alternatively we could have
>   sigreturn.c pass different versions of the sigreturn functions to
>   the kernel depending on which syscall ABI is in use at runtime, but
>   this requires bolting on new arch-specific logic to something that's
>   normally not arch-specific, so I prefer just having the conditionals
>   in the sigreturn functions.
> 
> - The syscall asm requires an extra register clobber to be able to do
>   the runtime trap number switching.
> 
> An alternate design I considered was simply patching the trap number
> in the .text for SH2, since there's no memory protection, but of
> course this would preclude execute-in-place from ROM, etc. so I don't
> think it's a proper solution. It is much less invasive, though.

We should finagle together an XIP test setup. There were two different
people doing it at ELC we could ask questions of.

>> 2. We need additional runtime detection options for atomics: interrupt
>>    masking for plain SH2, and the new CAS instruction for SH2J.
> 
> This is the one thing I haven't done, so currently the atomic macros
> are using GUSA which is non-atomic and unsafe on SH2 (if an interrupt
> happens with invalid stack pointer, memory will be corrupted).

And doesn't work with SMP at all.

> This
> could be part of the random crashing I've been experiencing (although
> I reproduced it without musl) so I'll try to add them next.

I'm going to try to post kernel patches to the list later today, and
separately email you the horrible ethernet drivers that aren't going
upstream because horrible. (We need to clean up the ethernet VHDL too,
it has timing issues that randomly work or don't work because layout
butterfly effects. Known problem, on the todo list, not my area...)

>> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
>>    won't work. I have a version locally but it doesn't make sense to
>>    commit without runtime trap number selection.
> 
> Done and updated to use runtime selection in the (ugly) patch.

If they ask for vfork() they should get vfork()...?

>> 4. As long as we're using the FDPIC ELF header flag to get
>>    binfmt_elf_fdpic.c to load binaries, the startup code needs to call
>>    the personality() syscall to switch back. I have a local hack for
>>    doing this in rcrt1.o which is probably not worth upstreaming if we
>>    can just make the kernel do it right.
> 
> No longer needed because of the kernel patch to load normal ELF.

Send me the patch and I'll add it to my stack to go upstream.

>> 5. The brk workaround I'm doing now can't be upstreamed without a
>>    reliable runtime way to distinguish nommu. To put it in malloc.c
>>    this would have to be a cross-arch solution. What might make more
>>    sense is putting it in syscall_arch.h for sh, where we already
>>    have to check for SH2 to determine the right trap number; the
>>    inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;
> 
> Commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 in musl solves this in
> a general manner, even though it's no longer needed with my kernel
> patch applied.

No longer needed on sh2 maybe, but there are a half-dozen other nommu
targets of interest...

> One more musl-side issue I neglected to mention is the __unmapself.s
> can't work on SH2 because the SH2 trap/interrupt mechanism requires
> the userspace stack pointer to be valid at all times. This is now
> solved upstream in commit c30cbcb0a646b1f13a22c645616dce624465b883,
> but activating it for SH2 requires removing
> src/thread/sh/__unmapself.s so the generic C file gets used.
> 
> The attached patch covers everything described above that's not
> already upstream, and is sufficient to build musl for sh2 with
> musl-cross targeting "sheb-linux-musl". I used gcc 4.7.3 because later
> versions break the kernel. The attached config.mak for musl shows the
> configure options I used. The attached sheb.specs file is how I got
> gcc to do always-PIE without breaking the kernel.

For the newly cc'd the relevant web archive entry is:

http://www.openwall.com/lists/musl/2015/06/10/1

Rob


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-11  4:02   ` Rob Landley
@ 2015-06-11 15:12     ` Rich Felker
  2015-06-11 17:22       ` Rich Felker
  2015-06-12  4:08       ` Yoshinori Sato
  0 siblings, 2 replies; 17+ messages in thread
From: Rich Felker @ 2015-06-11 15:12 UTC (permalink / raw)
  To: musl; +Cc: D. Jeff Dionne, ysato, shumpei.kawasaki

On Wed, Jun 10, 2015 at 11:02:35PM -0500, Rob Landley wrote:
> 
> 
> On 06/09/2015 10:30 PM, Rich Felker wrote:
> > On Mon, Jun 01, 2015 at 11:11:07AM -0400, Rich Felker wrote:
> >> [resent to musl list]
> >>
> >> Here's a summary of the issues we need to work through to get a modern
> >> SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
> >> and to the point where it's something people can use roughly 'out of
> >> the box':
> >>
> >> Kernel issues:
> >>
> >> 1. Kernel should support loading plain ELF directly, unmodified. Right
> > 
> > I have a patch to do this, not polished but it works. It also...
> > 
> >> 2. Kernel insists on having a stack size set in the PT_GNU_STACK
> >>    program header; if it's 0 (the default ld produces) then execve
> >>    fails. It should just provide a default, probably 128k (equal to
> >>    MMU-ful Linux).
> > 
> > ...uses a default stack size of 128k if the header is 0, and...
> 
> That's big enough to give a false sense of security without being big
> enough to actually work reliably on software that isn't thinking about it.

It's the same size MMU-ful Linux gives you, and it works for the vast
majority of applications. Of course with MMU you have the option to
expand on faults, but I think the only application I've ever seen do
this is emacs, and it typically only uses 164k. If you want to survey
existing applications this is easy to do using busybox "top" in "s"
mode, sorted by stack size. I wouldn't object to making the default
slightly larger if measurement shows it's needed.

Keep in mind that the purpose here is running binaries which are _not_
NOMMU-specific and not tuned for NOMMU. If your binaries are
NOMMU-specific than you choose a proper stack size.

> The kernel stack size is apparently 4k, but they've got more than one
> for different contexts:
> 
> https://www.kernel.org/doc/Documentation/x86/x86_64/kernel-stacks

Kernel stack is completely different. Userspace will not run with a 4k
stack if you use libc in any nontrivial way.

> I think I agree to jeff here: the historical default stack size for
> nommu packaging was 8k. That's enough for hello world to work, but in
> this context stack is a resource we need to allocate if we want
> nontrivial amounts of it.

That's doable (though still a bad idea, I think) as a default for a
NOMMU toolchain. It's not okay for running a binary that's not made
for NOMMU. This is not SH--specific but something that affects normal
ELF binaries running on any NOMMU machine, and it's an issue I'm going
to be adamant on, because adding the ability to do this with a huge
security flaw is just irresponsible.

> >> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
> >>    is that these two SH2A hardware traps overlap with the syscall
> >>    range used by SH3/4 ABI:
> > 
> > I haven't patched this yet. I'd like to use 31 (0x1f) as the new
> > universal SH syscall trap number, instead of 22. More details on the
> > reasons later.
> 
> I've cc'd Yoshinori Sato (who did most of the historical sh2 work) and
> Shumpei Kawasaki (the original superh architect). They'll probably have
> an opinion on your "more reasons" for changing sh2 system call numbers
> to match sh4.

Thank you. I'd really like to make progress at least on the matter of
determining if this is feasible. I now have a new musl/sh2 patch that
simply uses "trapa #31" unconditionally, and it's a lot
simpler/cleaner and working on my patched kernel. The big question is
just whether this is an unacceptable constraint on hardware.

> >> musl issues:
> >>
> >> 1. We need runtime detection for the right trap number to use for
> >>    syscalls. Right now I've got the trap numbers hard-coded for SH2 in
> >>    my local tree.
> > 
> > I've written the runtime detection, but I'd rather not have to use it.
> > I managed to avoid inlining a big conditional at each syscall, but
> > there are still multiple ugly issues:
> 
> This is only an issue if you want sh4 to be able to run sh2 binaries.

Yes, and I've explained why this is important. In summary:

- It lets you do initial testing (or even native compiling) on the
  MMU-ful variant of the target where you have things like memory
  protection to assist you in debugging and the ability to use a lot
  of software that wouldn't work on NOMMU.

- It fights the combinatoric explosion of configuration/target
  variants that made uclibc so difficult to maintain by minimizing the
  number of differences and by allowing regression testing without
  specialized hardware (testing can be done on real hardware or on
  qemu-system-sh4eb).

> (Can m68k run coldfire binaries? Last I checked there wasn't any
> blackfin-with-mmu variant.)

As long as you use an ISA level that's a subset of both, there's no
reason for it not to work. Demonstrating it without getting a common
libc working on both would just be a matter of making a -nostdlib PoC
program doing its own syscalls.

> > [...]
> 
> We should finagle together an XIP test setup. There were two different
> people doing it at ELC we could ask questions of.

From the userspace side (libc/startfiles/toolchain/etc.) XIP should
just work once we can make binaries in a format that allows for
shareable text. XIP-capable hardware isn't really needed to test this
side, just the kernel side.

> >> 2. We need additional runtime detection options for atomics: interrupt
> >>    masking for plain SH2, and the new CAS instruction for SH2J.
> > 
> > This is the one thing I haven't done, so currently the atomic macros
> > are using GUSA which is non-atomic and unsafe on SH2 (if an interrupt
> > happens with invalid stack pointer, memory will be corrupted).
> 
> And doesn't work with SMP at all.

Right. I just added interrupt-masking-based atomics for SH2 on my
side, but I know that's not useful for your SMP setups. It is useful
for older non-SMP SH2 hardware, though.

> > This
> > could be part of the random crashing I've been experiencing (although
> > I reproduced it without musl) so I'll try to add them next.
> 
> I'm going to try to post kernel patches to the list later today, and
> separately email you the horrible ethernet drivers that aren't going
> upstream because horrible. (We need to clean up the ethernet VHDL too,
> it has timing issues that randomly work or don't work because layout
> butterfly effects. Known problem, on the todo list, not my area...)

OK thanks!

> >> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> >>    won't work. I have a version locally but it doesn't make sense to
> >>    commit without runtime trap number selection.
> > 
> > Done and updated to use runtime selection in the (ugly) patch.
> 
> If they ask for vfork() they should get vfork()...?

Yes. The "runtime selection" is about the syscall trap number, not
whether or not to use vfork. I committed vfork to upstream musl now,
but with a SH3/4 trap number to be consistent with the code that's
upstream now. Later I'll either convert them all to trap 31 (0x1f) if
that ends up being acceptable, or merge the runtime-selection code,
but I think it makes sense to make the change across all files at
once, whichever way it's done.

> >> 4. As long as we're using the FDPIC ELF header flag to get
> >>    binfmt_elf_fdpic.c to load binaries, the startup code needs to call
> >>    the personality() syscall to switch back. I have a local hack for
> >>    doing this in rcrt1.o which is probably not worth upstreaming if we
> >>    can just make the kernel do it right.
> > 
> > No longer needed because of the kernel patch to load normal ELF.
> 
> Send me the patch and I'll add it to my stack to go upstream.

It was attached, but it may need a little bit more cleanup before
going upstream.

> >> 5. The brk workaround I'm doing now can't be upstreamed without a
> >>    reliable runtime way to distinguish nommu. To put it in malloc.c
> >>    this would have to be a cross-arch solution. What might make more
> >>    sense is putting it in syscall_arch.h for sh, where we already
> >>    have to check for SH2 to determine the right trap number; the
> >>    inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;
> > 
> > Commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 in musl solves this in
> > a general manner, even though it's no longer needed with my kernel
> > patch applied.
> 
> No longer needed on sh2 maybe, but there are a half-dozen other nommu
> targets of interest...

The change is not sh-specific. This is in the fdpic elf loader code
which is arch-generic. So if we can get the fix upstreamed, the issue
won't matter on any future kernel versions, but it's still good for
musl to be safe against being run on old kernels.

> > One more musl-side issue I neglected to mention is the __unmapself.s
> > can't work on SH2 because the SH2 trap/interrupt mechanism requires
> > the userspace stack pointer to be valid at all times. This is now
> > solved upstream in commit c30cbcb0a646b1f13a22c645616dce624465b883,
> > but activating it for SH2 requires removing
> > src/thread/sh/__unmapself.s so the generic C file gets used.
> > 
> > The attached patch covers everything described above that's not
> > already upstream, and is sufficient to build musl for sh2 with
> > musl-cross targeting "sheb-linux-musl". I used gcc 4.7.3 because later
> > versions break the kernel. The attached config.mak for musl shows the
> > configure options I used. The attached sheb.specs file is how I got
> > gcc to do always-PIE without breaking the kernel.
> 
> For the newly cc'd the relevant web archive entry is:
> 
> http://www.openwall.com/lists/musl/2015/06/10/1

Thanks!

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-11 15:12     ` Rich Felker
@ 2015-06-11 17:22       ` Rich Felker
  2015-06-12  4:26         ` Yoshinori Sato
  2015-06-12  4:08       ` Yoshinori Sato
  1 sibling, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-11 17:22 UTC (permalink / raw)
  To: musl; +Cc: D. Jeff Dionne, ysato, shumpei.kawasaki

On Thu, Jun 11, 2015 at 11:12:52AM -0400, Rich Felker wrote:
> > >> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> > >>    won't work. I have a version locally but it doesn't make sense to
> > >>    commit without runtime trap number selection.
> > > 
> > > Done and updated to use runtime selection in the (ugly) patch.
> > 
> > If they ask for vfork() they should get vfork()...?
> 
> Yes. The "runtime selection" is about the syscall trap number, not
> whether or not to use vfork. I committed vfork to upstream musl now,
> but with a SH3/4 trap number to be consistent with the code that's
> upstream now. Later I'll either convert them all to trap 31 (0x1f) if
> that ends up being acceptable, or merge the runtime-selection code,
> but I think it makes sense to make the change across all files at
> once, whichever way it's done.

Ah, maybe I misunderstood. If you were asking abaout the original
remark that the default vfork.c uses fork, the reason is simply that
you can't write vfork() in C. The return from vfork() in the child
will clobber vfork's stack frame, which may contain the return address
or saved registers, and then when the parent resumes, very bad things
will happen. vfork() has to be implemented in asm to ensure that any
state it needs to be able to return in the parent is kept in registers
rather than memory. Thus, each arch needs an arch-specific version,
and we just hadn't gotten around to adding the sh version yet.

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-11 15:12     ` Rich Felker
  2015-06-11 17:22       ` Rich Felker
@ 2015-06-12  4:08       ` Yoshinori Sato
  2015-06-12  4:28         ` Rich Felker
  1 sibling, 1 reply; 17+ messages in thread
From: Yoshinori Sato @ 2015-06-12  4:08 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Fri, 12 Jun 2015 00:12:52 +0900,
Rich Felker wrote:
> 
> On Wed, Jun 10, 2015 at 11:02:35PM -0500, Rob Landley wrote:
> > 
> > 
> > On 06/09/2015 10:30 PM, Rich Felker wrote:
> > > On Mon, Jun 01, 2015 at 11:11:07AM -0400, Rich Felker wrote:
> > >> [resent to musl list]
> > >>
> > >> Here's a summary of the issues we need to work through to get a modern
> > >> SH2/nommu-targetted musl/toolchain out of the proof-of-concept stage
> > >> and to the point where it's something people can use roughly 'out of
> > >> the box':
> > >>
> > >> Kernel issues:
> > >>
> > >> 1. Kernel should support loading plain ELF directly, unmodified. Right
> > > 
> > > I have a patch to do this, not polished but it works. It also...
> > > 
> > >> 2. Kernel insists on having a stack size set in the PT_GNU_STACK
> > >>    program header; if it's 0 (the default ld produces) then execve
> > >>    fails. It should just provide a default, probably 128k (equal to
> > >>    MMU-ful Linux).
> > > 
> > > ...uses a default stack size of 128k if the header is 0, and...
> > 
> > That's big enough to give a false sense of security without being big
> > enough to actually work reliably on software that isn't thinking about it.
> 
> It's the same size MMU-ful Linux gives you, and it works for the vast
> majority of applications. Of course with MMU you have the option to
> expand on faults, but I think the only application I've ever seen do
> this is emacs, and it typically only uses 164k. If you want to survey
> existing applications this is easy to do using busybox "top" in "s"
> mode, sorted by stack size. I wouldn't object to making the default
> slightly larger if measurement shows it's needed.
> 
> Keep in mind that the purpose here is running binaries which are _not_
> NOMMU-specific and not tuned for NOMMU. If your binaries are
> NOMMU-specific than you choose a proper stack size.
> 
> > The kernel stack size is apparently 4k, but they've got more than one
> > for different contexts:
> > 
> > https://www.kernel.org/doc/Documentation/x86/x86_64/kernel-stacks
> 
> Kernel stack is completely different. Userspace will not run with a 4k
> stack if you use libc in any nontrivial way.
> 
> > I think I agree to jeff here: the historical default stack size for
> > nommu packaging was 8k. That's enough for hello world to work, but in
> > this context stack is a resource we need to allocate if we want
> > nontrivial amounts of it.
> 
> That's doable (though still a bad idea, I think) as a default for a
> NOMMU toolchain. It's not okay for running a binary that's not made
> for NOMMU. This is not SH--specific but something that affects normal
> ELF binaries running on any NOMMU machine, and it's an issue I'm going
> to be adamant on, because adding the ability to do this with a huge
> security flaw is just irresponsible.
> 
> > >> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
> > >>    is that these two SH2A hardware traps overlap with the syscall
> > >>    range used by SH3/4 ABI:
> > > 
> > > I haven't patched this yet. I'd like to use 31 (0x1f) as the new
> > > universal SH syscall trap number, instead of 22. More details on the
> > > reasons later.
> > 
> > I've cc'd Yoshinori Sato (who did most of the historical sh2 work) and
> > Shumpei Kawasaki (the original superh architect). They'll probably have
> > an opinion on your "more reasons" for changing sh2 system call numbers
> > to match sh4.

It histrical reason.
SH3/4 is assigned #0x10 to #0x17 for system call entry.
But SH2A system using this vector.
So we moved to #0x20 to #0x27 for SH2A.
(SH2A specification is #0x20 to #0x3f allocated for user application.)

And SH2 port is based on SH2A port.
It have same systemcall interface.

> Thank you. I'd really like to make progress at least on the matter of
> determining if this is feasible. I now have a new musl/sh2 patch that
> simply uses "trapa #31" unconditionally, and it's a lot
> simpler/cleaner and working on my patched kernel. The big question is
> just whether this is an unacceptable constraint on hardware.

SH2A reserved system for vector 31.
But not assigned now.
I think no problem.

> > >> musl issues:
> > >>
> > >> 1. We need runtime detection for the right trap number to use for
> > >>    syscalls. Right now I've got the trap numbers hard-coded for SH2 in
> > >>    my local tree.
> > > 
> > > I've written the runtime detection, but I'd rather not have to use it.
> > > I managed to avoid inlining a big conditional at each syscall, but
> > > there are still multiple ugly issues:
> > 
> > This is only an issue if you want sh4 to be able to run sh2 binaries.
> 
> Yes, and I've explained why this is important. In summary:
> 
> - It lets you do initial testing (or even native compiling) on the
>   MMU-ful variant of the target where you have things like memory
>   protection to assist you in debugging and the ability to use a lot
>   of software that wouldn't work on NOMMU.
> 
> - It fights the combinatoric explosion of configuration/target
>   variants that made uclibc so difficult to maintain by minimizing the
>   number of differences and by allowing regression testing without
>   specialized hardware (testing can be done on real hardware or on
>   qemu-system-sh4eb).
> 
> > (Can m68k run coldfire binaries? Last I checked there wasn't any
> > blackfin-with-mmu variant.)
> 
> As long as you use an ISA level that's a subset of both, there's no
> reason for it not to work. Demonstrating it without getting a common
> libc working on both would just be a matter of making a -nostdlib PoC
> program doing its own syscalls.
> 
> > > [...]
> > 
> > We should finagle together an XIP test setup. There were two different
> > people doing it at ELC we could ask questions of.
> 
> From the userspace side (libc/startfiles/toolchain/etc.) XIP should
> just work once we can make binaries in a format that allows for
> shareable text. XIP-capable hardware isn't really needed to test this
> side, just the kernel side.
> 
> > >> 2. We need additional runtime detection options for atomics: interrupt
> > >>    masking for plain SH2, and the new CAS instruction for SH2J.
> > > 
> > > This is the one thing I haven't done, so currently the atomic macros
> > > are using GUSA which is non-atomic and unsafe on SH2 (if an interrupt
> > > happens with invalid stack pointer, memory will be corrupted).
> > 
> > And doesn't work with SMP at all.
> 
> Right. I just added interrupt-masking-based atomics for SH2 on my
> side, but I know that's not useful for your SMP setups. It is useful
> for older non-SMP SH2 hardware, though.
> 
> > > This
> > > could be part of the random crashing I've been experiencing (although
> > > I reproduced it without musl) so I'll try to add them next.
> > 
> > I'm going to try to post kernel patches to the list later today, and
> > separately email you the horrible ethernet drivers that aren't going
> > upstream because horrible. (We need to clean up the ethernet VHDL too,
> > it has timing issues that randomly work or don't work because layout
> > butterfly effects. Known problem, on the todo list, not my area...)
> 
> OK thanks!
> 
> > >> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> > >>    won't work. I have a version locally but it doesn't make sense to
> > >>    commit without runtime trap number selection.
> > > 
> > > Done and updated to use runtime selection in the (ugly) patch.
> > 
> > If they ask for vfork() they should get vfork()...?
> 
> Yes. The "runtime selection" is about the syscall trap number, not
> whether or not to use vfork. I committed vfork to upstream musl now,
> but with a SH3/4 trap number to be consistent with the code that's
> upstream now. Later I'll either convert them all to trap 31 (0x1f) if
> that ends up being acceptable, or merge the runtime-selection code,
> but I think it makes sense to make the change across all files at
> once, whichever way it's done.
> 
> > >> 4. As long as we're using the FDPIC ELF header flag to get
> > >>    binfmt_elf_fdpic.c to load binaries, the startup code needs to call
> > >>    the personality() syscall to switch back. I have a local hack for
> > >>    doing this in rcrt1.o which is probably not worth upstreaming if we
> > >>    can just make the kernel do it right.
> > > 
> > > No longer needed because of the kernel patch to load normal ELF.
> > 
> > Send me the patch and I'll add it to my stack to go upstream.
> 
> It was attached, but it may need a little bit more cleanup before
> going upstream.
> 
> > >> 5. The brk workaround I'm doing now can't be upstreamed without a
> > >>    reliable runtime way to distinguish nommu. To put it in malloc.c
> > >>    this would have to be a cross-arch solution. What might make more
> > >>    sense is putting it in syscall_arch.h for sh, where we already
> > >>    have to check for SH2 to determine the right trap number; the
> > >>    inline syscall code can just do if (nr==SYS_brk&&IS_SH2) return 0;
> > > 
> > > Commit 276904c2f6bde3a31a24ebfa201482601d18b4f9 in musl solves this in
> > > a general manner, even though it's no longer needed with my kernel
> > > patch applied.
> > 
> > No longer needed on sh2 maybe, but there are a half-dozen other nommu
> > targets of interest...
> 
> The change is not sh-specific. This is in the fdpic elf loader code
> which is arch-generic. So if we can get the fix upstreamed, the issue
> won't matter on any future kernel versions, but it's still good for
> musl to be safe against being run on old kernels.
> 
> > > One more musl-side issue I neglected to mention is the __unmapself.s
> > > can't work on SH2 because the SH2 trap/interrupt mechanism requires
> > > the userspace stack pointer to be valid at all times. This is now
> > > solved upstream in commit c30cbcb0a646b1f13a22c645616dce624465b883,
> > > but activating it for SH2 requires removing
> > > src/thread/sh/__unmapself.s so the generic C file gets used.
> > > 
> > > The attached patch covers everything described above that's not
> > > already upstream, and is sufficient to build musl for sh2 with
> > > musl-cross targeting "sheb-linux-musl". I used gcc 4.7.3 because later
> > > versions break the kernel. The attached config.mak for musl shows the
> > > configure options I used. The attached sheb.specs file is how I got
> > > gcc to do always-PIE without breaking the kernel.
> > 
> > For the newly cc'd the relevant web archive entry is:
> > 
> > http://www.openwall.com/lists/musl/2015/06/10/1
> 
> Thanks!
> 
> Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-11 17:22       ` Rich Felker
@ 2015-06-12  4:26         ` Yoshinori Sato
  2015-06-12  4:35           ` Rich Felker
  0 siblings, 1 reply; 17+ messages in thread
From: Yoshinori Sato @ 2015-06-12  4:26 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Fri, 12 Jun 2015 02:22:27 +0900,
Rich Felker wrote:
> 
> On Thu, Jun 11, 2015 at 11:12:52AM -0400, Rich Felker wrote:
> > > >> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> > > >>    won't work. I have a version locally but it doesn't make sense to
> > > >>    commit without runtime trap number selection.
> > > > 
> > > > Done and updated to use runtime selection in the (ugly) patch.
> > > 
> > > If they ask for vfork() they should get vfork()...?
> > 
> > Yes. The "runtime selection" is about the syscall trap number, not
> > whether or not to use vfork. I committed vfork to upstream musl now,
> > but with a SH3/4 trap number to be consistent with the code that's
> > upstream now. Later I'll either convert them all to trap 31 (0x1f) if
> > that ends up being acceptable, or merge the runtime-selection code,
> > but I think it makes sense to make the change across all files at
> > once, whichever way it's done.
> 
> Ah, maybe I misunderstood. If you were asking abaout the original
> remark that the default vfork.c uses fork, the reason is simply that
> you can't write vfork() in C. The return from vfork() in the child
> will clobber vfork's stack frame, which may contain the return address
> or saved registers, and then when the parent resumes, very bad things
> will happen. vfork() has to be implemented in asm to ensure that any
> state it needs to be able to return in the parent is kept in registers
> rather than memory. Thus, each arch needs an arch-specific version,
> and we just hadn't gotten around to adding the sh version yet.
> 
> Rich

No. vfork kept only last return address.
It isn't necessary to preserve the value of anything but that.
Child process can't return caller routine.

-- 
Yoshinori Sato
<ysato@users.sourceforge.jp>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  4:08       ` Yoshinori Sato
@ 2015-06-12  4:28         ` Rich Felker
  2015-06-16  6:38           ` Yoshinori Sato
  0 siblings, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-12  4:28 UTC (permalink / raw)
  To: Yoshinori Sato; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Fri, Jun 12, 2015 at 01:08:05PM +0900, Yoshinori Sato wrote:
> > > >> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
> > > >>    is that these two SH2A hardware traps overlap with the syscall
> > > >>    range used by SH3/4 ABI:
> > > > 
> > > > I haven't patched this yet. I'd like to use 31 (0x1f) as the new
> > > > universal SH syscall trap number, instead of 22. More details on the
> > > > reasons later.
> > > 
> > > I've cc'd Yoshinori Sato (who did most of the historical sh2 work) and
> > > Shumpei Kawasaki (the original superh architect). They'll probably have
> > > an opinion on your "more reasons" for changing sh2 system call numbers
> > > to match sh4.
> 
> It histrical reason.
> SH3/4 is assigned #0x10 to #0x17 for system call entry.
> But SH2A system using this vector.
> So we moved to #0x20 to #0x27 for SH2A.
> (SH2A specification is #0x20 to #0x3f allocated for user application.)
> 
> And SH2 port is based on SH2A port.
> It have same systemcall interface.
> 
> > Thank you. I'd really like to make progress at least on the matter of
> > determining if this is feasible. I now have a new musl/sh2 patch that
> > simply uses "trapa #31" unconditionally, and it's a lot
> > simpler/cleaner and working on my patched kernel. The big question is
> > just whether this is an unacceptable constraint on hardware.
> 
> SH2A reserved system for vector 31.
> But not assigned now.
> I think no problem.

Thank you for the feedback. This sounds promising.

We still need whoever ends up being the new kernel maintainer for SH
to be okay with adding trap 31 syscall support for sh2 and declaring
it supported/stable for sh3/4 too, but at least it looks like there
arent technical problems for doing this.

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  4:26         ` Yoshinori Sato
@ 2015-06-12  4:35           ` Rich Felker
  2015-06-12  4:49             ` uClinux.org
  0 siblings, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-12  4:35 UTC (permalink / raw)
  To: Yoshinori Sato; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Fri, Jun 12, 2015 at 01:26:55PM +0900, Yoshinori Sato wrote:
> On Fri, 12 Jun 2015 02:22:27 +0900,
> Rich Felker wrote:
> > 
> > On Thu, Jun 11, 2015 at 11:12:52AM -0400, Rich Felker wrote:
> > > > >> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> > > > >>    won't work. I have a version locally but it doesn't make sense to
> > > > >>    commit without runtime trap number selection.
> > > > > 
> > > > > Done and updated to use runtime selection in the (ugly) patch.
> > > > 
> > > > If they ask for vfork() they should get vfork()...?
> > > 
> > > Yes. The "runtime selection" is about the syscall trap number, not
> > > whether or not to use vfork. I committed vfork to upstream musl now,
> > > but with a SH3/4 trap number to be consistent with the code that's
> > > upstream now. Later I'll either convert them all to trap 31 (0x1f) if
> > > that ends up being acceptable, or merge the runtime-selection code,
> > > but I think it makes sense to make the change across all files at
> > > once, whichever way it's done.
> > 
> > Ah, maybe I misunderstood. If you were asking abaout the original
> > remark that the default vfork.c uses fork, the reason is simply that
> > you can't write vfork() in C. The return from vfork() in the child
> > will clobber vfork's stack frame, which may contain the return address
> > or saved registers, and then when the parent resumes, very bad things
> > will happen. vfork() has to be implemented in asm to ensure that any
> > state it needs to be able to return in the parent is kept in registers
> > rather than memory. Thus, each arch needs an arch-specific version,
> > and we just hadn't gotten around to adding the sh version yet.
> 
> No. vfork kept only last return address.
> It isn't necessary to preserve the value of anything but that.
> Child process can't return caller routine.

vfork still has to follow the normal function call ABI of preserving
call-saved registers. For example, if you (or the compiler) wrote
vfork by spilling some or all of the call-saved registers to the
stack, clobbering them (e.g. for stack-protector work, or profiling
counters, or PIC-related purposes, or for no reason at all), and then
restoring them at return time, you'd be in trouble. The first return
(in the child) would properly restore these registers, but subsequent
execution in the child (in the function that called vfork, e.g. when
it sets up the stack for a call to execl) could clobber the locations
where they were saved on the stack, and when the parent resumed
execution, it vfork would restore the wrong values, and very bad
things could happen in the caller (e.g. the GOT register used for
loading string literal args to exec*() might be wrong).

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  4:35           ` Rich Felker
@ 2015-06-12  4:49             ` uClinux.org
  2015-06-12  6:37               ` Rich Felker
  0 siblings, 1 reply; 17+ messages in thread
From: uClinux.org @ 2015-06-12  4:49 UTC (permalink / raw)
  To: Rich Felker; +Cc: Yoshinori Sato, musl, shumpei.kawasaki

Rich,

This isn't necessary.  The child in a nommu system -must never- return from the function that called vfork.  The reason is there is only one stack, and so that stack must not be corrupted (not the pointer, the actual call frame) or as you mention the parent cannot continue execution.

This is not a nommu uClinux thing, it is a restriction we inherited from BSD vfork().  It makes things much simpler (read: tractable at all), actually.

Cheers,
J.

> On Jun 12, 2015, at 13:35, Rich Felker <dalias@libc.org> wrote:
> 
>> On Fri, Jun 12, 2015 at 01:26:55PM +0900, Yoshinori Sato wrote:
>> On Fri, 12 Jun 2015 02:22:27 +0900,
>> Rich Felker wrote:
>>> 
>>> On Thu, Jun 11, 2015 at 11:12:52AM -0400, Rich Felker wrote:
>>>>>>> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
>>>>>>>   won't work. I have a version locally but it doesn't make sense to
>>>>>>>   commit without runtime trap number selection.
>>>>>> 
>>>>>> Done and updated to use runtime selection in the (ugly) patch.
>>>>> 
>>>>> If they ask for vfork() they should get vfork()...?
>>>> 
>>>> Yes. The "runtime selection" is about the syscall trap number, not
>>>> whether or not to use vfork. I committed vfork to upstream musl now,
>>>> but with a SH3/4 trap number to be consistent with the code that's
>>>> upstream now. Later I'll either convert them all to trap 31 (0x1f) if
>>>> that ends up being acceptable, or merge the runtime-selection code,
>>>> but I think it makes sense to make the change across all files at
>>>> once, whichever way it's done.
>>> 
>>> Ah, maybe I misunderstood. If you were asking abaout the original
>>> remark that the default vfork.c uses fork, the reason is simply that
>>> you can't write vfork() in C. The return from vfork() in the child
>>> will clobber vfork's stack frame, which may contain the return address
>>> or saved registers, and then when the parent resumes, very bad things
>>> will happen. vfork() has to be implemented in asm to ensure that any
>>> state it needs to be able to return in the parent is kept in registers
>>> rather than memory. Thus, each arch needs an arch-specific version,
>>> and we just hadn't gotten around to adding the sh version yet.
>> 
>> No. vfork kept only last return address.
>> It isn't necessary to preserve the value of anything but that.
>> Child process can't return caller routine.
> 
> vfork still has to follow the normal function call ABI of preserving
> call-saved registers. For example, if you (or the compiler) wrote
> vfork by spilling some or all of the call-saved registers to the
> stack, clobbering them (e.g. for stack-protector work, or profiling
> counters, or PIC-related purposes, or for no reason at all), and then
> restoring them at return time, you'd be in trouble. The first return
> (in the child) would properly restore these registers, but subsequent
> execution in the child (in the function that called vfork, e.g. when
> it sets up the stack for a call to execl) could clobber the locations
> where they were saved on the stack, and when the parent resumed
> execution, it vfork would restore the wrong values, and very bad
> things could happen in the caller (e.g. the GOT register used for
> loading string literal args to exec*() might be wrong).
> 
> Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  4:49             ` uClinux.org
@ 2015-06-12  6:37               ` Rich Felker
  2015-06-12  6:46                 ` D. Jeff Dionne
  0 siblings, 1 reply; 17+ messages in thread
From: Rich Felker @ 2015-06-12  6:37 UTC (permalink / raw)
  To: uClinux.org; +Cc: Yoshinori Sato, musl, shumpei.kawasaki

On Fri, Jun 12, 2015 at 01:49:28PM +0900, uClinux.org wrote:
> Rich,
> 
> This isn't necessary. The child in a nommu system -must never-
> return from the function that called vfork. The reason is there is
> only one stack, and so that stack must not be corrupted (not the
> pointer, the actual call frame) or as you mention the parent cannot
> continue execution.
> 
> This is not a nommu uClinux thing, it is a restriction we inherited
> from BSD vfork(). It makes things much simpler (read: tractable at
> all), actually.

I'm not talking about returning from the function that called vfork.
This is about returning from vfork itself, to the caller of vfork.

Rich

> > On Jun 12, 2015, at 13:35, Rich Felker <dalias@libc.org> wrote:
> > 
> >> On Fri, Jun 12, 2015 at 01:26:55PM +0900, Yoshinori Sato wrote:
> >> On Fri, 12 Jun 2015 02:22:27 +0900,
> >> Rich Felker wrote:
> >>> 
> >>> On Thu, Jun 11, 2015 at 11:12:52AM -0400, Rich Felker wrote:
> >>>>>>> 3. We need sh/vfork.s since the default vfork.c just uses fork, which
> >>>>>>>   won't work. I have a version locally but it doesn't make sense to
> >>>>>>>   commit without runtime trap number selection.
> >>>>>> 
> >>>>>> Done and updated to use runtime selection in the (ugly) patch.
> >>>>> 
> >>>>> If they ask for vfork() they should get vfork()...?
> >>>> 
> >>>> Yes. The "runtime selection" is about the syscall trap number, not
> >>>> whether or not to use vfork. I committed vfork to upstream musl now,
> >>>> but with a SH3/4 trap number to be consistent with the code that's
> >>>> upstream now. Later I'll either convert them all to trap 31 (0x1f) if
> >>>> that ends up being acceptable, or merge the runtime-selection code,
> >>>> but I think it makes sense to make the change across all files at
> >>>> once, whichever way it's done.
> >>> 
> >>> Ah, maybe I misunderstood. If you were asking abaout the original
> >>> remark that the default vfork.c uses fork, the reason is simply that
> >>> you can't write vfork() in C. The return from vfork() in the child
> >>> will clobber vfork's stack frame, which may contain the return address
> >>> or saved registers, and then when the parent resumes, very bad things
> >>> will happen. vfork() has to be implemented in asm to ensure that any
> >>> state it needs to be able to return in the parent is kept in registers
> >>> rather than memory. Thus, each arch needs an arch-specific version,
> >>> and we just hadn't gotten around to adding the sh version yet.
> >> 
> >> No. vfork kept only last return address.
> >> It isn't necessary to preserve the value of anything but that.
> >> Child process can't return caller routine.
> > 
> > vfork still has to follow the normal function call ABI of preserving
> > call-saved registers. For example, if you (or the compiler) wrote
> > vfork by spilling some or all of the call-saved registers to the
> > stack, clobbering them (e.g. for stack-protector work, or profiling
> > counters, or PIC-related purposes, or for no reason at all), and then
> > restoring them at return time, you'd be in trouble. The first return
> > (in the child) would properly restore these registers, but subsequent
> > execution in the child (in the function that called vfork, e.g. when
> > it sets up the stack for a call to execl) could clobber the locations
> > where they were saved on the stack, and when the parent resumed
> > execution, it vfork would restore the wrong values, and very bad
> > things could happen in the caller (e.g. the GOT register used for
> > loading string literal args to exec*() might be wrong).
> > 
> > Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  6:37               ` Rich Felker
@ 2015-06-12  6:46                 ` D. Jeff Dionne
  0 siblings, 0 replies; 17+ messages in thread
From: D. Jeff Dionne @ 2015-06-12  6:46 UTC (permalink / raw)
  To: Rich Felker; +Cc: D. Jeff Dionne, Yoshinori Sato, musl, shumpei.kawasaki


> On Jun 12, 2015, at 3:37 PM, Rich Felker <dalias@libc.org> wrote:
>> 
>> This is not a nommu uClinux thing, it is a restriction we inherited
>> from BSD vfork(). It makes things much simpler (read: tractable at
>> all), actually.
> 
> I'm not talking about returning from the function that called vfork.
> This is about returning from vfork itself, to the caller of vfork.

>>>  The first return
>>> (in the child) would properly restore these registers, but subsequent
>>> execution in the child (in the function that called vfork, e.g. when
>>> it sets up the stack for a call to execl) could clobber the locations
>>> where they were saved on the stack, and when the parent resumed
>>> execution, it vfork would restore the wrong values, and very bad
>>> things could happen in the caller

Oh, I see what you were saying, sorry about that.  You are correct, there
can be no C stack frame for vfork(), I agree.

Cheers,
J.



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-12  4:28         ` Rich Felker
@ 2015-06-16  6:38           ` Yoshinori Sato
  2015-06-16  7:02             ` Rich Felker
  0 siblings, 1 reply; 17+ messages in thread
From: Yoshinori Sato @ 2015-06-16  6:38 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Fri, 12 Jun 2015 13:28:21 +0900,
Rich Felker wrote:
> 
> On Fri, Jun 12, 2015 at 01:08:05PM +0900, Yoshinori Sato wrote:
> > > > >> 4. Syscall trap numbers differ on SH2 vs SH3/4. Presumably the reason
> > > > >>    is that these two SH2A hardware traps overlap with the syscall
> > > > >>    range used by SH3/4 ABI:
> > > > > 
> > > > > I haven't patched this yet. I'd like to use 31 (0x1f) as the new
> > > > > universal SH syscall trap number, instead of 22. More details on the
> > > > > reasons later.
> > > > 
> > > > I've cc'd Yoshinori Sato (who did most of the historical sh2 work) and
> > > > Shumpei Kawasaki (the original superh architect). They'll probably have
> > > > an opinion on your "more reasons" for changing sh2 system call numbers
> > > > to match sh4.
> > 
> > It histrical reason.
> > SH3/4 is assigned #0x10 to #0x17 for system call entry.
> > But SH2A system using this vector.
> > So we moved to #0x20 to #0x27 for SH2A.
> > (SH2A specification is #0x20 to #0x3f allocated for user application.)
> > 
> > And SH2 port is based on SH2A port.
> > It have same systemcall interface.
> > 
> > > Thank you. I'd really like to make progress at least on the matter of
> > > determining if this is feasible. I now have a new musl/sh2 patch that
> > > simply uses "trapa #31" unconditionally, and it's a lot
> > > simpler/cleaner and working on my patched kernel. The big question is
> > > just whether this is an unacceptable constraint on hardware.
> > 
> > SH2A reserved system for vector 31.
> > But not assigned now.
> > I think no problem.
> 
> Thank you for the feedback. This sounds promising.
> 
> We still need whoever ends up being the new kernel maintainer for SH
> to be okay with adding trap 31 syscall support for sh2 and declaring
> it supported/stable for sh3/4 too, but at least it looks like there
> arent technical problems for doing this.
> 
> Rich

Yes.
I think test necessary by SH2A, but there would be no problems.

-- 
Yoshinori Sato
<ysato@users.sourceforge.jp>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Moving forward with sh2/nommu
  2015-06-16  6:38           ` Yoshinori Sato
@ 2015-06-16  7:02             ` Rich Felker
  0 siblings, 0 replies; 17+ messages in thread
From: Rich Felker @ 2015-06-16  7:02 UTC (permalink / raw)
  To: Yoshinori Sato; +Cc: musl, D. Jeff Dionne, shumpei.kawasaki

On Tue, Jun 16, 2015 at 03:38:59PM +0900, Yoshinori Sato wrote:
> > > > Thank you. I'd really like to make progress at least on the matter of
> > > > determining if this is feasible. I now have a new musl/sh2 patch that
> > > > simply uses "trapa #31" unconditionally, and it's a lot
> > > > simpler/cleaner and working on my patched kernel. The big question is
> > > > just whether this is an unacceptable constraint on hardware.
> > > 
> > > SH2A reserved system for vector 31.
> > > But not assigned now.
> > > I think no problem.
> > 
> > Thank you for the feedback. This sounds promising.
> > 
> > We still need whoever ends up being the new kernel maintainer for SH
> > to be okay with adding trap 31 syscall support for sh2 and declaring
> > it supported/stable for sh3/4 too, but at least it looks like there
> > arent technical problems for doing this.
> 
> Yes.
> I think test necessary by SH2A, but there would be no problems.

Do you know anyone who could verify that vector 31 is unused? There's
nothing in the upstream Linux kernel attached to it, but I'm not clear
on whether somebody would have to make a new SH2A chip variant
(unlikely) to use it for hardware purposes, or whether it could come
into use as a result of how an existing SH2A chip is wired up to other
hardware on the board.

Rich


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2015-06-16  7:02 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-01 15:11 Moving forward with sh2/nommu Rich Felker
2015-06-02  6:09 ` Rob Landley
2015-06-02 16:45   ` Rich Felker
2015-06-02 23:49     ` Rich Felker
2015-06-10  3:30 ` Rich Felker
2015-06-11  4:02   ` Rob Landley
2015-06-11 15:12     ` Rich Felker
2015-06-11 17:22       ` Rich Felker
2015-06-12  4:26         ` Yoshinori Sato
2015-06-12  4:35           ` Rich Felker
2015-06-12  4:49             ` uClinux.org
2015-06-12  6:37               ` Rich Felker
2015-06-12  6:46                 ` D. Jeff Dionne
2015-06-12  4:08       ` Yoshinori Sato
2015-06-12  4:28         ` Rich Felker
2015-06-16  6:38           ` Yoshinori Sato
2015-06-16  7:02             ` Rich Felker

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).