porting musl to RISCV-64

porting musl to RISCV-64

[Gry Gunvor]

I am interested in compiling for RISCV-64 (http://riscv.org/) using a
libc that I compile myself so that I can run it through some
source-to-source transformations before compiling it.  The default
RISCV tools will target their ports of either gnu libc or newlib.  Gnu
libc is huge, so I would rather avoid it; also I would rather use
something released under a permissive open source license (BSD).
Newlib is small, but I am not interested in multithreading and the
multithreading/reentrancy support they have built into the library is
causing me problems.  So I thought I would try musl.

I'm running on a vanilla Ubuntu 14.04 x86 machine and muscl builds out
of the box.  Using your gcc wrapper, hello world links and runs.  I
tried wrapping malloc at link time and that worked.  Thus, so far,
muscl is working great.

FYI: You might want to know that this recent post on the subject of
teaching C speaks of musl as "high-quality":
http://blog.regehr.org/archives/1393
"We’ll want some lecture material about C’s place in the modern world
and we also need to spend time reading some high-quality C code,
perhaps starting with Redis, Musl, or Xv6. Musl, in particular, is a
good match for teaching since it contains lots of cute little
functions that can be understood in isolation."

Anyway, suppose I want to port musl to RISCV-64 and cross-compile it.
MIPS is a RISC architecture that is rather close to RISCV, but your
MIPS support says 32-bit (and big-endian), so I take it you have some
dependencies on what used to be called the "machine word size" (RISCV
is 64-bit, but the RISCV docs still call a 32-bit unit a "word"; ugh;
so now we need a new word for "word").  Thus I suspect that this is
going to be non-trival.  How much work is this going to be?  Hints?

More generally, any comments on what do you recommend as the easiest
thing to do?  Port musl to RISCV-64?  Use a different libc?  For those
of you who know newlib, hack newlib to remove reentrancy goop?

Gry

Re: porting musl to RISCV-64

[Szabolcs Nagy]

* Gry Gunvor <gry.gunvor@gmail.com> [2016-05-10 23:05:01 -0700]:
> Newlib is small, but I am not interested in multithreading and the
> multithreading/reentrancy support they have built into the library is
> causing me problems.  So I thought I would try musl.
> 

newlib is for baremetal development, you don't
need to use threading with it, but on linux i
think it uses the mostly broken linuxthreads.

musl is a conforming posix libc so it has
complete pthreads support.

> Anyway, suppose I want to port musl to RISCV-64 and cross-compile it.
> MIPS is a RISC architecture that is rather close to RISCV, but your
> MIPS support says 32-bit (and big-endian), so I take it you have some
> dependencies on what used to be called the "machine word size" (RISCV
> is 64-bit, but the RISCV docs still call a 32-bit unit a "word"; ugh;
> so now we need a new word for "word").  Thus I suspect that this is
> going to be non-trival.  How much work is this going to be?  Hints?
> 

musl now has 32bit mips, mips64 and mipsn32 support as well.
these are different abis so they have to be separate ports,
same is true for the riscv targets.

there is a google summer of code project to add riscv support
http://www.openwall.com/lists/musl/2016/04/27/3
i think it is supposed to provide a working port within 1-2
months (?) so if you can wait you don't need to do much work.

> More generally, any comments on what do you recommend as the easiest
> thing to do?  Port musl to RISCV-64?  Use a different libc?  For those
> of you who know newlib, hack newlib to remove reentrancy goop?
> 

i don't know what you are trying to do so it's hard to tell,
if you want a complete userspace on linux then i think a newlib
subset is not enough, but if you want to get some specific
application to run then it might work.

Re: porting musl to RISCV-64

[Gry Gunvor]

On Wed, May 11, 2016 at 4:25 AM, Szabolcs Nagy <nsz@port70.net> wrote:

> newlib is for baremetal development, you don't
> need to use threading with it

It's attempt at reentrancy support is causing me problems.

> musl now has 32bit mips, mips64 and mipsn32 support as well.
> these are different abis so they have to be separate ports,
> same is true for the riscv targets.

Your documentation does not seem to mention the MIPS64 port, but now
that I look in arch, I see the directory for it.

> there is a google summer of code project to add riscv support
> http://www.openwall.com/lists/musl/2016/04/27/3
> i think it is supposed to provide a working port within 1-2
> months (?) so if you can wait you don't need to do much work.

Suppose I can't wait and I attempt this myself.  Right now I'm just
trying to get a generic libc working.  I do not want to handle
multi-threading or signals.  What can I omit?  Are the
non-portabilities isolated in arch/ ?  That is, is there much more to
it than cloning the arch/mips64 directory and hacking on it?

atomic_arch.h: I think I can make all of these functions empty as I am
not going to be using multi-threading, right?

crt_arch.h: program startup; what is this doing in a libc
implementation?  doesn't the compiler handle this?

ksigaction.h: sorry, I'm not a hardware person; I suppose different
hardware has a different default layout for a signal object? so this
is not a thing determined by kernel software?  I don't care to handle
signals right now anyway.

pthread_arch.h: again, I think I can make all of these functions empty
as I am not going to be using multi-threading.

reloc.h: I can't figure out what this is.

syscall_arch.h: I've already written this for RISCV-64 (and so have
the RISCV people).

bits: to what extent is this MIPS64-specific?  since there is *no*
inline assembly, how come arch/generic/ doesn't do here?

Gry

Re: porting musl to RISCV-64

[Gry Gunvor]

Here's another way to put my question: I've already written my own
ghetto riscv-64 libc for synchronous programming (no threads, signals)
by writing my own version of what you call syscall_arch.h.  Therefore,
how hard would it be to factor musl so that syscall_arch.h was all I
was depending on that is arch dependent?  It seems that should get me
a lot of what libc provides.  I would want that if I attempt to
compile a program that does more than that, say, attempts to use
threads/signals/something-weird that I just get a compile-time error.
Here's a naive plan:

 * grep for the inclusion of the other header files in arch other than
syscall_arch.h;

 * for .c files, comment them out in the makefile;

 * for .h files, insert an #error directive.

What will go wrong?

Gry


On Wed, May 11, 2016 at 9:32 AM, Gry Gunvor <gry.gunvor@gmail.com> wrote:
> On Wed, May 11, 2016 at 4:25 AM, Szabolcs Nagy <nsz@port70.net> wrote:
>
>> newlib is for baremetal development, you don't
>> need to use threading with it
>
> It's attempt at reentrancy support is causing me problems.
>
>> musl now has 32bit mips, mips64 and mipsn32 support as well.
>> these are different abis so they have to be separate ports,
>> same is true for the riscv targets.
>
> Your documentation does not seem to mention the MIPS64 port, but now
> that I look in arch, I see the directory for it.
>
>> there is a google summer of code project to add riscv support
>> http://www.openwall.com/lists/musl/2016/04/27/3
>> i think it is supposed to provide a working port within 1-2
>> months (?) so if you can wait you don't need to do much work.
>
> Suppose I can't wait and I attempt this myself.  Right now I'm just
> trying to get a generic libc working.  I do not want to handle
> multi-threading or signals.  What can I omit?  Are the
> non-portabilities isolated in arch/ ?  That is, is there much more to
> it than cloning the arch/mips64 directory and hacking on it?
>
> atomic_arch.h: I think I can make all of these functions empty as I am
> not going to be using multi-threading, right?
>
> crt_arch.h: program startup; what is this doing in a libc
> implementation?  doesn't the compiler handle this?
>
> ksigaction.h: sorry, I'm not a hardware person; I suppose different
> hardware has a different default layout for a signal object? so this
> is not a thing determined by kernel software?  I don't care to handle
> signals right now anyway.
>
> pthread_arch.h: again, I think I can make all of these functions empty
> as I am not going to be using multi-threading.
>
> reloc.h: I can't figure out what this is.
>
> syscall_arch.h: I've already written this for RISCV-64 (and so have
> the RISCV people).
>
> bits: to what extent is this MIPS64-specific?  since there is *no*
> inline assembly, how come arch/generic/ doesn't do here?
>
> Gry

Re: porting musl to RISCV-64

[Szabolcs Nagy]

* Gry Gunvor <gry.gunvor@gmail.com> [2016-05-11 09:40:42 -0700]:
> Here's another way to put my question: I've already written my own
> ghetto riscv-64 libc for synchronous programming (no threads, signals)
> by writing my own version of what you call syscall_arch.h.  Therefore,
> how hard would it be to factor musl so that syscall_arch.h was all I
> was depending on that is arch dependent?  It seems that should get me

the previous riscv port attempt might be enough for you
https://github.com/lluixhi/musl-riscv
but i've never used any of the riscv stuff.

> a lot of what libc provides.  I would want that if I attempt to
> compile a program that does more than that, say, attempts to use
> threads/signals/something-weird that I just get a compile-time error.
> Here's a naive plan:
> 
>  * grep for the inclusion of the other header files in arch other than
> syscall_arch.h;
> 
>  * for .c files, comment them out in the makefile;
> 
>  * for .h files, insert an #error directive.
> 
> What will go wrong?
> 

there are lot of subtle issues that may or may not matter for you.

> Gry
> 
> 
> On Wed, May 11, 2016 at 9:32 AM, Gry Gunvor <gry.gunvor@gmail.com> wrote:
> > On Wed, May 11, 2016 at 4:25 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> >
> >> newlib is for baremetal development, you don't
> >> need to use threading with it
> >
> > It's attempt at reentrancy support is causing me problems.
> >
> >> musl now has 32bit mips, mips64 and mipsn32 support as well.
> >> these are different abis so they have to be separate ports,
> >> same is true for the riscv targets.
> >
> > Your documentation does not seem to mention the MIPS64 port, but now
> > that I look in arch, I see the directory for it.
> >
> >> there is a google summer of code project to add riscv support
> >> http://www.openwall.com/lists/musl/2016/04/27/3
> >> i think it is supposed to provide a working port within 1-2
> >> months (?) so if you can wait you don't need to do much work.
> >
> > Suppose I can't wait and I attempt this myself.  Right now I'm just
> > trying to get a generic libc working.  I do not want to handle
> > multi-threading or signals.  What can I omit?  Are the
> > non-portabilities isolated in arch/ ?  That is, is there much more to
> > it than cloning the arch/mips64 directory and hacking on it?
> >
> > atomic_arch.h: I think I can make all of these functions empty as I am
> > not going to be using multi-threading, right?
> >
> > crt_arch.h: program startup; what is this doing in a libc
> > implementation?  doesn't the compiler handle this?
> >
> > ksigaction.h: sorry, I'm not a hardware person; I suppose different
> > hardware has a different default layout for a signal object? so this
> > is not a thing determined by kernel software?  I don't care to handle
> > signals right now anyway.
> >
> > pthread_arch.h: again, I think I can make all of these functions empty
> > as I am not going to be using multi-threading.
> >
> > reloc.h: I can't figure out what this is.
> >
> > syscall_arch.h: I've already written this for RISCV-64 (and so have
> > the RISCV people).
> >
> > bits: to what extent is this MIPS64-specific?  since there is *no*
> > inline assembly, how come arch/generic/ doesn't do here?
> >
> > Gry

Re: porting musl to RISCV-64

[Bobby Bingham]

On Wed, May 11, 2016 at 09:32:16AM -0700, Gry Gunvor wrote:
> On Wed, May 11, 2016 at 4:25 AM, Szabolcs Nagy <nsz@port70.net> wrote:
>
> > newlib is for baremetal development, you don't
> > need to use threading with it
>
> It's attempt at reentrancy support is causing me problems.
>
> > musl now has 32bit mips, mips64 and mipsn32 support as well.
> > these are different abis so they have to be separate ports,
> > same is true for the riscv targets.
>
> Your documentation does not seem to mention the MIPS64 port, but now
> that I look in arch, I see the directory for it.

This is a pretty new port.  The documentation just hasn't been updated
yet.

>
> > there is a google summer of code project to add riscv support
> > http://www.openwall.com/lists/musl/2016/04/27/3
> > i think it is supposed to provide a working port within 1-2
> > months (?) so if you can wait you don't need to do much work.
>
> Suppose I can't wait and I attempt this myself.  Right now I'm just
> trying to get a generic libc working.  I do not want to handle
> multi-threading or signals.  What can I omit?  Are the
> non-portabilities isolated in arch/ ?  That is, is there much more to
> it than cloning the arch/mips64 directory and hacking on it?

There are some bits of architecture-specific assembly scattered
elsewhere (crti/crtn, setjmp/longjmp, syscall, syscall_cp).

The __clone function which is used to create new threads is also used
internally to implement a couple other functions that aren't strictly
threading-related (eg, posix_spawn and a fallback version of faccessat),
so if you omit it those functions may also not be usable.

>
> atomic_arch.h: I think I can make all of these functions empty as I am
> not going to be using multi-threading, right?

These functions don't really need to be atomic for a single thread to
work, but they do still need to perform the correct operations.  If you're
ignoring the atomicity required for multi-threaded code, these are all
trivial to implement in C.

>
> crt_arch.h: program startup; what is this doing in a libc
> implementation?  doesn't the compiler handle this?

This has the entry point that is invoked by the kernel, and handles the
little bit of setup necessary for handing off control to C code.

>
> ksigaction.h: sorry, I'm not a hardware person; I suppose different
> hardware has a different default layout for a signal object? so this
> is not a thing determined by kernel software?  I don't care to handle
> signals right now anyway.

This needs to match the kernel.  Not all musl ports have this file.
Some use the generic version in src/internal/ksigaction.h

>
> pthread_arch.h: again, I think I can make all of these functions empty
> as I am not going to be using multi-threading.

This is code for working with the thread pointer and thread-local storage.
Even if you only have a single thread, this is still necessary.  For
example, errno is thread-local and is accessed through the thread
pointer (see the definition of __errno_location).

>
> reloc.h: I can't figure out what this is.

It's definitions for the dynamic linker.

>
> syscall_arch.h: I've already written this for RISCV-64 (and so have
> the RISCV people).
>
> bits: to what extent is this MIPS64-specific?  since there is *no*
> inline assembly, how come arch/generic/ doesn't do here?

A small amount of it depends on the hardware (fenv) or on the ABI
(alltypes).  Most of it though just needs to match the kernel, and the
Linux kernel definitions vary by architecture.

--
Bobby