GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

[Roman Titov]

Proposal submited to google-melange:
http://www.google-melange.com/gsoc/proposal/public/google/gsoc2015/zoorg/5629499534213120

I'm still working on it, making a lot of fixes and small additions.
Any feedback appreciated, as always.

Regards,
Roman Titov

GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

[Roman Titov]

[-- Attachment #1: Type: text/plain, Size: 450 bytes --]

2/3 completed proposal attached. Too tired to finish it completelly 
today.
It's said that good proposal should be exact, realistic and brief...
Well, it's not. It's my first GSoC, so I think the more, the better.

Any kind of feedback appreciated.
Following kinds of feedback is GREATLY appreciated:
1. Milestones and timeline. All your thoughts about them.
2. What should be shortened and/or deleted? Any better formulations?

Regards,
Roman Titov

[-- Attachment #2: lowRISC proposal draft.txt --]
[-- Type: text/plain, Size: 5717 bytes --]

Porting musl libc to RISC-V


Description:
...


Goals and deliverables:
* Milestone #1 (MANDATORY):
    Complete musl-libc port for riscv-linux, targeting RV32G and RV64G ISAs.

    Milestone #1 completion criterions:
    Functional criterion:
        riscv-musl (lets call it so) should pass all libc-test [http://wiki.musl-libc.org/wiki/Libc-Test] tests, except tests for:
        * optional features that are not implemented in musl yet (e.g. posix/XSI, mentioned at libc-test musl-wiki page above)
        * features, that could not be implemented, because of ISA limitations
        * features, that could not be implemented, because of riscv-toolkit limitations (e.g. unimplemented riscv-gcc or riscv-linux feature/API or unstable riscv-gcc or riscv-linux API/ABI)

        Of course reasoned explanation of "why particular feature was not implemented" should (and would) be granted.

    Perfomance criterion:
        riscv-musl should be tested with libc-bench [http://www.etalabs.net/libc-bench.html][http://git.musl-libc.org/cgit/libc-bench/] benchmark and should not show any significant regression compared to existing riscv-musl benchmarking results [http://www.etalabs.net/compare_libcs.html] for other platforms.

* Milestone #2 (MANDATORY):
    *Analysis* and/or testing of potential optimization candidates. What, why and how could be further optimized and which perfomance increase expected.

* Milestone #3 (OPTIONAL):
    Implementation (partial or full) of *found* optimizations with constant regression and functional *testing*.

If all milestones would be achieved ahead of time, or as a further development of the project after GSoC completion, following stretch goal could be implemented:
    Adapt riscv-musl to modular nature of RISC-V ISA, e.g. adapt build system for modular ISA (conditional compilation and stuff), add support for soft-float, for ISA's without F and D extensions, add support for no-mmu systems, etc.


Timeline:
This timeline targets more of pessimistically-realistic schedule.
Porting libc to the new, mostly undocumented ISA, is hard and routine task, so I include a lot of time for investigation, taking wrong pathes, etc.

As I'm a pre-graduaded student, I won't be able to work full-time during June. So, I would spread my June work to "community bonding" (i.e. before 25.05.2015, when GSoC coding officially starts) period as well.

30.03.2015 - 26.04.2015
    Catching up with my university stuff.
    0-15 hours of work per week.

    My main milestone for this period is:
    Investigating source code, specifying list of things that I don't know or don't understand yet. As a result by the end of April I should clearly "know the things that I don't know".
    e.g. Rich Felker mentions "cas" in his musl-libc porting tips [http://www.openwall.com/lists/musl/2012/07/08/1]; RISC-V ISA also mentions "CAS" and says that RISC-V implements "LR/SC" approach. And I have no idea what both of them means and how it affects my work.

    Other milestones:
    Porting crt.
    Getting familiar with arch/*/bits, start porting process.

    This is not very tough work (we don't even know, if I'm accepted into GSoC until 27.04.2015), yet it's extremely important.

27.04.2015 - 03.05.2015 ("Community bonding" period)
    University.
    15-30 hours of work.

    Main milestone for this week is:
    Make up detailed schedule for following 2 months.

    And of course continue previous work in some reasonable way.

04.05.2015 - 24.05.2015 ("Community bonding" period)
    University.
    15-30 hours of work per week.

    Main milestone for this period is:
    Achieve requirments for "most single-threaded, static-linked programs", according to Rich Felker's porting tips [http://www.openwall.com/lists/musl/2012/07/08/1].

    Other tasks:
    Get familiar with build system.
    Get fully working and ready development environment. Toolchain. ISA simulator. At least *begin* working on testsuites (libc-test and libc-bench).
    e.g. libc-test use make for both building suite and runing tests, so either I should have working riscv-linux image with gcc and make or I should prepare some cross-testing suite.

25.05.2015 - 31.05.2015 (GSoC start)
    Exams.
    0-20 hours of work.

    Work continued in the same way.
    Dynamic linking, threading and everything else.

01.06.2015 - 07.06.2015
    Exams.
    0-20 hours of work.

    ...

08.06.2015 - 14.06.2015
    Almost free.
    *At least* 20-30 hours of work.

    ...

15.06.2015 - 21.06.2015
    Almost free.
    *At least* 20-30 hours of work.

    ...

22.06.2015 - 28.06.2015 (GSoC mid-term evaluation; from 26.06.2015)
    Evaluation work verification. Degree.
    15-30 hours of work.

    Work continued in the same way.
    Intensive testing and bugfixing.

29.06.2015 - 05.07.2015 (GSoC mid-term evaluation; till 03.07.2015)
    Full 40 hours of work per week *from now on*.

    Work continued in the same way.
    Dynamic linking, threading and everything else.
    Intensive testing and bugfixing.

    *Milestone #1 should be fully reached this week!*

06.07.2015 - 12.07.2015
    Working on milestones #2 and #3 from now on.
    Constant testing and bugfixing.

13.07.2015 - 19.07.2015
    ...

20.07.2015 - 26.07.2015
    ...

27.07.2015 - 02.08.2015
    ...

03.08.2015 - 09.08.2015
    ...

10.08.2015 - 16.08.2015
    ...

17.08.2015 - 23.08.2015 (GSoC finish)
    Final rounds of testing, bugfixing and code-polishing.
    Working on documentation and reports.


Why this project?:
...


Personal details:
Roman Titov
email: titovroman@gmail.com
IRC: zoorg
Phone number: +7(914)326-28-26
Postal adress:
55a-51 Russkaya Str.
Vladivostok Primorskiy region 690105
Russian Federation

Re: GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

[Roman Titov]

2015-03-26 15:11 GMT+10:00 Rich Felker <dalias@libc.org>:
>
> Can you clarify the G suffix?
>

RISC-V ISA is modular.
Specification says that only base integer subset is mandatory.
At the moment there is RV32I or RV64I bases, and RV128I possible
somwhere in the future. Integer base bitness defines overall bitness
of resulting ISA, obviously.
Above integer base there is also a set of "standard extensions":
  M - integer multiplication and division,
  A - atomic instructions,
  F - single-precision floating point,
  D - double-precision floating point.

Direct quote from ISA spec (last sentence on page 4 continued on page 5):
"An integer base plus these four standard extensions (“IMAFD”) is
given the abbreviation “G” and provides a
general-purpose scalar instruction set. RV32G and RV64G are currently
the default target of our
compiler toolchains."

I think this also answers one of your next questions:

>
> This should not include anything except fenv on soft-float archs. I'm
> not clear on whether the proposed port is soft-float, hard-float, or
> both, and whether there would be separate ABIs for both or just one
> ABI.
>

G is a default target, so I think we also targeting it.

Regards,
Roman Titov

Re: GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

[Rich Felker]

On Thu, Mar 26, 2015 at 01:12:03PM +1000, Roman Titov wrote:
> Hello, musl.
> 
> At the moment I'm working on pre-final form of my proposal and I
> need some help here.
> I defined 3 milestones for the project:
> 
> * Milestone #1 (MANDATORY):
>    Complete musl-libc port for riscv-linux, targeting RV32G and
> RV64G ISAs.

Can you clarify the G suffix?

> * Milestone #2 (MANDATORY):
>    *Analysis* and/or testing of potential optimization candidates.
> What, why and how could be further optimized and which perfomance
> increase expected.
> 
> * Milestone #3 (OPTIONAL):
>    Implementation (partial or full) of *found* optimizations with
> constant functional and regression *testing*.
> 
> I need help with exact criterion definitions for milestone #1.
> "Complete musl-libc port..." is nothing. We can't rate level of
> success or failure here, as no obligations given and so, this is not
> a good proposal milestone.

Roughly it should mean a working, non-stub version of each mandatory
arch-specific file musl needs, where working can be demonstrated in
some way. For functionality not already covered by one of the tests in
libc-test, it may be nice to add tests to libc-test as a means of
demonstrating that there are no obvious failures.

> ***
> So, here are my suggestions:
> 
> Functional criterion:
> riscv-musl (lets call it so) should pass all libc-test
> [http://wiki.musl-libc.org/wiki/Libc-Test] tests, except tests for:
> * optional features that are not implemented in musl yet (e.g.
> posix/XSI, mentioned at libc-test musl-wiki page above)

At his point I don't think there are any optional features tested that
aren't implemented. There are a few mandatory macros that aren't
defined in musl's headers due to lack of an established value they
should have but these failures are obvious. The rest of the current
failures on "working" archs are minor math issues and known bugs with
pending fixes. Aside perhaps from math issues where the C code is less
accurate than existing x86 asm, a new "working" arch should not have
failures that don't also show up on x86.

> * features, that could not be implemented, because of ISA limitations

This should not include anything except fenv on soft-float archs. I'm
not clear on whether the proposed port is soft-float, hard-float, or
both, and whether there would be separate ABIs for both or just one
ABI.

> * features, that could not be implemented, because of riscv-toolkit
> limitations (e.g. unimplemented riscv-gcc or riscv-linux feature/API
> or unstable riscv-gcc or riscv-linux API/ABI)

I don't believe anything like that will exist. Do you have examples in
mind?

> Of course reasoned explanation of "why particular feature was not
> implemented" should (and would) be granted.

Yes, if there's a reason something can't be done then it would be
reasonable to document that. But I don't anticipate hitting such
issues anyway.

> Perfomance criterion:
> riscv-musl should be tested with libc-bench [http://www.etalabs.net/libc-bench.html][http://git.musl-libc.org/cgit/libc-bench/]
> benchmark and should not show any significant regression compared to
> existing riscv-musl benchmarking results
> [http://www.etalabs.net/compare_libcs.html] for other platforms.

It might make sense to do some new timing tests, especially as part of
an evaluation of where further work is needed.

> (Right now I still have no better perfomance criterion and no better
> definition of "significant regression" than "intuitive" one.
> Generally, my idea is to compare riscv-musl libc-bench results with
> existing musl implementations and with existing riscv-glibc.
> riscv-musl and musl results should correlate, if any riscv-glibc
> results is 2 times better - investigate.)
> ***

Do you know if testing on real hardware will be possible, or if the
emulator can simulate "real hardware" timing.

> Any feedback is highly appreciated.
> Is this milestones *ok* and *enough for the job*? Do you *agree*
> with them? (probably not so significant to musl, as for lowRISC,
> which is actual project "owner")
> Are my criterions *ok*? Do you *agree* with them?
> *Any* additions, deletions, corrections, advices, etc.
> 
> "Full" version of proposal would be in a few hours.

As I mentioned on IRC I'm still travelling from ELC right now so I
haven't had a chance to review and comment to the extent I'd like.
I'll try to give more feedback tomorrow.

Rich

GSoC 2015. Porting musl libc to RISC-V project. Proposal help and feedback.

[Roman Titov]

Hello, musl.

At the moment I'm working on pre-final form of my proposal and I need 
some help here.
I defined 3 milestones for the project:

* Milestone #1 (MANDATORY):
    Complete musl-libc port for riscv-linux, targeting RV32G and RV64G 
ISAs.

* Milestone #2 (MANDATORY):
    *Analysis* and/or testing of potential optimization candidates. 
What, why and how could be further optimized and which perfomance 
increase expected.

* Milestone #3 (OPTIONAL):
    Implementation (partial or full) of *found* optimizations with 
constant functional and regression *testing*.

I need help with exact criterion definitions for milestone #1. 
"Complete musl-libc port..." is nothing. We can't rate level of success 
or failure here, as no obligations given and so, this is not a good 
proposal milestone.

***
So, here are my suggestions:

Functional criterion:
riscv-musl (lets call it so) should pass all libc-test 
[http://wiki.musl-libc.org/wiki/Libc-Test] tests, except tests for:
* optional features that are not implemented in musl yet (e.g. 
posix/XSI, mentioned at libc-test musl-wiki page above)
* features, that could not be implemented, because of ISA limitations
* features, that could not be implemented, because of riscv-toolkit 
limitations (e.g. unimplemented riscv-gcc or riscv-linux feature/API or 
unstable riscv-gcc or riscv-linux API/ABI)

Of course reasoned explanation of "why particular feature was not 
implemented" should (and would) be granted.

Perfomance criterion:
riscv-musl should be tested with libc-bench 
[http://www.etalabs.net/libc-bench.html][http://git.musl-libc.org/cgit/libc-bench/] 
benchmark and should not show any significant regression compared to 
existing riscv-musl benchmarking results 
[http://www.etalabs.net/compare_libcs.html] for other platforms.

(Right now I still have no better perfomance criterion and no better 
definition of "significant regression" than "intuitive" one. Generally, 
my idea is to compare riscv-musl libc-bench results with existing musl 
implementations and with existing riscv-glibc. riscv-musl and musl 
results should correlate, if any riscv-glibc results is 2 times better 
- investigate.)
***

Any feedback is highly appreciated.
Is this milestones *ok* and *enough for the job*? Do you *agree* with 
them? (probably not so significant to musl, as for lowRISC, which is 
actual project "owner")
Are my criterions *ok*? Do you *agree* with them?
*Any* additions, deletions, corrections, advices, etc.

"Full" version of proposal would be in a few hours.

Regards,
Roman Titov