musl risc-v port & gsoc - resources & ideas

musl risc-v port & gsoc - resources & ideas

[Rich Felker]

lowrisc.org has been accepted into Google Summer of Code 2016, and has
porting musl to risc-v as one of the suggested projects:

http://www.lowrisc.org/docs/gsoc-2016-ideas/

I'm very hopeful that we'll make the port happen this year. In this
email I'd like to go over some resources that may be helpful to
students interested in applying, and some ideas for other tasks that
could be included in proposals.

The musl wiki contains a porting page with some useful but
not-entirely-up-to-date information on porting musl to a new arch.
This is a good starting point, and updating it could actually be part
of the gsoc project. See http://wiki.musl-libc.org/wiki/Porting

Some information on recent changes can be found in the mailing list
archives. These threads pertain to changes to how ports are expected
to provide atomic primitives:

http://www.openwall.com/lists/musl/2015/05/17/2
http://www.openwall.com/lists/musl/2015/05/20/1
http://www.openwall.com/lists/musl/2016/01/10/6

which was committed here:

http://git.musl-libc.org/cgit/musl/commit/?id=1315596b510189b5159e742110b504177bdd4932

and other subsequent commits with per-arch improvements.

And these cover the bits deduplication:

http://www.openwall.com/lists/musl/2016/01/25/1
http://www.openwall.com/lists/musl/2016/01/27/9

which was committed here:

http://git.musl-libc.org/cgit/musl/commit/?id=4dfac11538cb20c848c30d754863800061ee8c81

Threads on the recent mips64 port work, which is almost ready for
merging, may also be helpful to read. It's broken up across several
threads but you can find most of the content in the January-March 2016
archives.

Since a port of musl to a new arch does not actually involve much
code, mainly attention to detail to make sure that all of the type
definitions/ABI/etc. are correct, I think that for a proposal to be
big enough to make a reasonable GSoC project, it should go beyond just
the basic porting. Some ideas for things to include would be:

- Improvement of porting documentation

- Feedback/patches on where there's too much redundancy between ports
  and how to reduce it (i.e. making improvements to musl that reduce
  the amount of code/headers needed for a new port).

- Patches for musl-cross and/or musl-cross-make (build systems for
  generating a cross-compiler toolchain) to make it easy to build a
  musl/riscv cross compiler.

- Optimizing performance-critical code like memcpy or floating point
  math functions for riscv.

- Improving test coverage, especially for things that are easy to get
  wrong in a new port.

I'll follow up with more ideas if I think of any.

Students interested in the project are welcome (and encouraged!) to
ask questions and discuss here on the musl list. Obviously everyone
should have in mind writing their own proposals but I want everyone to
have access to knowledge/resources/community for ideas.

Rich

Re: musl risc-v port & gsoc - resources & ideas

[Rich Felker]

On Thu, Mar 03, 2016 at 10:58:12PM -0500, Rich Felker wrote:
> lowrisc.org has been accepted into Google Summer of Code 2016, and has
> porting musl to risc-v as one of the suggested projects:
> 
> http://www.lowrisc.org/docs/gsoc-2016-ideas/
> 
> I'm very hopeful that we'll make the port happen this year. In this
> email I'd like to go over some resources that may be helpful to
> students interested in applying, and some ideas for other tasks that
> could be included in proposals.
> 
> The musl wiki contains a porting page with some useful but
> not-entirely-up-to-date information on porting musl to a new arch.
> This is a good starting point, and updating it could actually be part
> of the gsoc project. See http://wiki.musl-libc.org/wiki/Porting
> 
> Some information on recent changes can be found in the mailing list
> archives. These threads pertain to changes to how ports are expected
> to provide atomic primitives:
> 
> http://www.openwall.com/lists/musl/2015/05/17/2
> http://www.openwall.com/lists/musl/2015/05/20/1
> http://www.openwall.com/lists/musl/2016/01/10/6
> 
> which was committed here:
> 
> http://git.musl-libc.org/cgit/musl/commit/?id=1315596b510189b5159e742110b504177bdd4932
> 
> and other subsequent commits with per-arch improvements.
> 
> And these cover the bits deduplication:
> 
> http://www.openwall.com/lists/musl/2016/01/25/1
> http://www.openwall.com/lists/musl/2016/01/27/9
> 
> which was committed here:
> 
> http://git.musl-libc.org/cgit/musl/commit/?id=4dfac11538cb20c848c30d754863800061ee8c81
> 
> Threads on the recent mips64 port work, which is almost ready for
> merging, may also be helpful to read. It's broken up across several
> threads but you can find most of the content in the January-March 2016
> archives.
> 
> Since a port of musl to a new arch does not actually involve much
> code, mainly attention to detail to make sure that all of the type
> definitions/ABI/etc. are correct, I think that for a proposal to be
> big enough to make a reasonable GSoC project, it should go beyond just
> the basic porting. Some ideas for things to include would be:
> 
> - Improvement of porting documentation
> 
> - Feedback/patches on where there's too much redundancy between ports
>   and how to reduce it (i.e. making improvements to musl that reduce
>   the amount of code/headers needed for a new port).
> 
> - Patches for musl-cross and/or musl-cross-make (build systems for
>   generating a cross-compiler toolchain) to make it easy to build a
>   musl/riscv cross compiler.
> 
> - Optimizing performance-critical code like memcpy or floating point
>   math functions for riscv.
> 
> - Improving test coverage, especially for things that are easy to get
>   wrong in a new port.
> 
> I'll follow up with more ideas if I think of any.
> 
> Students interested in the project are welcome (and encouraged!) to
> ask questions and discuss here on the musl list. Obviously everyone
> should have in mind writing their own proposals but I want everyone to
> have access to knowledge/resources/community for ideas.

One more thing: the current test suite we use is here:

http://nsz.repo.hu/git/?p=libc-test

It replaces the old one on the musl git site, which has not been
updated in a long time and has nowhere near the coverage. Having this
could be useful to familiarize yourself with the types of testing that
we have and to get ideas for coverage areas you could propose to add.

Also, while we don't have any formal requirement for sample
problems/tasks to go with applications, looking for interesting bug
reports on the mailing list (or bugs fixes in musl's git log) and
writing some simple regression tests or broader tests of the affected
code to add to libc-test would be a nice way to demonstrate your
ability, especially if you don't already have existing FOSS project
contributions to cite, and would actually produce something of value.

Rich

Re: musl risc-v port & gsoc - resources & ideas

[Hesham Almatary]

Hi Rich,

Thanks for this detailed e-mail. It's worth mentioning that during my
last GSoC project that ported seL4 to RISC-V, I had to add RISC-V
support to the muslc library [1] (only 32-bit, imitating the or1k
port). It was mainly useful for userspace tasks, and works pretty
well.

I thought this might be a good starting point for anyone who might
work on this project. We can work to get this local code upstream if
that makes sense.

[1] https://github.com/heshamelmatary/musllibc

Best,
Hesham

On Fri, Mar 4, 2016 at 3:58 AM, Rich Felker <dalias@libc.org> wrote:
> lowrisc.org has been accepted into Google Summer of Code 2016, and has
> porting musl to risc-v as one of the suggested projects:
>
> http://www.lowrisc.org/docs/gsoc-2016-ideas/
>
> I'm very hopeful that we'll make the port happen this year. In this
> email I'd like to go over some resources that may be helpful to
> students interested in applying, and some ideas for other tasks that
> could be included in proposals.
>
> The musl wiki contains a porting page with some useful but
> not-entirely-up-to-date information on porting musl to a new arch.
> This is a good starting point, and updating it could actually be part
> of the gsoc project. See http://wiki.musl-libc.org/wiki/Porting
>
> Some information on recent changes can be found in the mailing list
> archives. These threads pertain to changes to how ports are expected
> to provide atomic primitives:
>
> http://www.openwall.com/lists/musl/2015/05/17/2
> http://www.openwall.com/lists/musl/2015/05/20/1
> http://www.openwall.com/lists/musl/2016/01/10/6
>
> which was committed here:
>
> http://git.musl-libc.org/cgit/musl/commit/?id=1315596b510189b5159e742110b504177bdd4932
>
> and other subsequent commits with per-arch improvements.
>
> And these cover the bits deduplication:
>
> http://www.openwall.com/lists/musl/2016/01/25/1
> http://www.openwall.com/lists/musl/2016/01/27/9
>
> which was committed here:
>
> http://git.musl-libc.org/cgit/musl/commit/?id=4dfac11538cb20c848c30d754863800061ee8c81
>
> Threads on the recent mips64 port work, which is almost ready for
> merging, may also be helpful to read. It's broken up across several
> threads but you can find most of the content in the January-March 2016
> archives.
>
> Since a port of musl to a new arch does not actually involve much
> code, mainly attention to detail to make sure that all of the type
> definitions/ABI/etc. are correct, I think that for a proposal to be
> big enough to make a reasonable GSoC project, it should go beyond just
> the basic porting. Some ideas for things to include would be:
>
> - Improvement of porting documentation
>
> - Feedback/patches on where there's too much redundancy between ports
>   and how to reduce it (i.e. making improvements to musl that reduce
>   the amount of code/headers needed for a new port).
>
> - Patches for musl-cross and/or musl-cross-make (build systems for
>   generating a cross-compiler toolchain) to make it easy to build a
>   musl/riscv cross compiler.
>
> - Optimizing performance-critical code like memcpy or floating point
>   math functions for riscv.
>
> - Improving test coverage, especially for things that are easy to get
>   wrong in a new port.
>
> I'll follow up with more ideas if I think of any.
>
> Students interested in the project are welcome (and encouraged!) to
> ask questions and discuss here on the musl list. Obviously everyone
> should have in mind writing their own proposals but I want everyone to
> have access to knowledge/resources/community for ideas.
>
> Rich
>



-- 
Hesham

Re: Re: [lowrisc-dev] musl risc-v port & gsoc - resources & ideas

[Rich Felker]

On Sun, Mar 06, 2016 at 06:31:30PM +0000, Hesham Almatary wrote:
> Hi Rich,
> 
> Thanks for this detailed e-mail. It's worth mentioning that during my
> last GSoC project that ported seL4 to RISC-V, I had to add RISC-V
> support to the muslc library [1] (only 32-bit, imitating the or1k
> port). It was mainly useful for userspace tasks, and works pretty
> well.
> 
> I thought this might be a good starting point for anyone who might
> work on this project. We can work to get this local code upstream if
> that makes sense.
> 
> [1] https://github.com/heshamelmatary/musllibc

Thanks for letting us know. Another student who's interested in the
project also just contacted me and has a partially-finished port that
was done independently. So I think a viable GSoC proposal for this is
going to need to go well beyond the initial porting work and aim to
get riscv support to first-class status. That probably means putting a
strong focus on testing, finding performance bottlenecks (which
doesn't necessarily mean writing asm to fix them; rather it might be
identifying and reporting compiler bugs that are causing the compiler
to generate bad code for the C), etc.

Now would be a good time for students interested in the project to
jump in with ideas.

Rich

Re: musl risc-v port & gsoc - resources & ideas

[Rich Felker]

On Thu, Mar 03, 2016 at 10:58:12PM -0500, Rich Felker wrote:
> Since a port of musl to a new arch does not actually involve much
> code, mainly attention to detail to make sure that all of the type
> definitions/ABI/etc. are correct, I think that for a proposal to be
> big enough to make a reasonable GSoC project, it should go beyond just
> the basic porting. Some ideas for things to include would be:
> 
> - Improvement of porting documentation
> 
> - Feedback/patches on where there's too much redundancy between ports
>   and how to reduce it (i.e. making improvements to musl that reduce
>   the amount of code/headers needed for a new port).
> 
> - Patches for musl-cross and/or musl-cross-make (build systems for
>   generating a cross-compiler toolchain) to make it easy to build a
>   musl/riscv cross compiler.
> 
> - Optimizing performance-critical code like memcpy or floating point
>   math functions for riscv.
> 
> - Improving test coverage, especially for things that are easy to get
>   wrong in a new port.
> 
> I'll follow up with more ideas if I think of any.
> 
> Students interested in the project are welcome (and encouraged!) to
> ask questions and discuss here on the musl list. Obviously everyone
> should have in mind writing their own proposals but I want everyone to
> have access to knowledge/resources/community for ideas.

One other thing I forgot: I believe the riscv ABIs use IEEE quad for
long double, but musl has very poor support for quad right now, Most
functions just call out to the double functions; only a few that
critically need to be exact actually do the right thing. Improving
this situation is important for making riscv and other archs (aarch64,
mips64) that use quad polished/first-class.

Also the lowrisc-dev cc seems to have gotten dropped on these emails
which are very relevant:

http://www.openwall.com/lists/musl/2016/03/06/17
http://www.openwall.com/lists/musl/2016/03/06/24
http://www.openwall.com/lists/musl/2016/03/06/25

Rich