Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

[Gardner, Ryan P]

Szabolcs Nagy <nsz@...t70.net> writes:

> thanks
> patches look good. but it will take a week or two
> for me to be able to test and apply them.

Thank you for the quick response. We appreciate your timeline for testing and applying the patches.

Our plan is to submit tests for the majority of functions that currently lack coverage within libc-test. 
We already have a significant number of tests written and ready for submission. To avoid overwhelming
you with patches, do you have a preferred cadence for the submission of these tests?

Additionally, we are planning to setup runtime tests in collaboration with the Buildroot community. This
would facilitate testing across various embedded configurations and processor architectures. Once up 
and running, when we submit new test patches we can reference a fork of Buildroot that shows the tests 
passing on various hardware configurations.

Looking forward to your guidance,
Ryan Gardner

Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

[Szabolcs Nagy]

* Gardner, Ryan P <ryan.p.gardner@boeing.com> [2024-11-07 04:39:59 +0000]:
> Szabolcs Nagy <nsz@...t70.net> writes:
> 
> > thanks
> > patches look good. but it will take a week or two
> > for me to be able to test and apply them.
> 
> Thank you for the quick response. We appreciate your timeline for testing and applying the patches.
> 
> Our plan is to submit tests for the majority of functions that currently lack coverage within libc-test. 
> We already have a significant number of tests written and ready for submission. To avoid overwhelming
> you with patches, do you have a preferred cadence for the submission of these tests?
> 

over the next month im only in front of a computer sporadically

but code style or quality of tests are less strict than for libc,
so i dont expect many iterations. just post the patches so when i
can take a look i can comment or apply them.

btw if there is a repo with a branch i can cherrypick from that
is a bit easier for me than handling mails. then i think you
dont even have to post "obvious" patches to the list. i can pick
them from the branch.

> Additionally, we are planning to setup runtime tests in collaboration with the Buildroot community. This
> would facilitate testing across various embedded configurations and processor architectures. Once up 
> and running, when we submit new test patches we can reference a fork of Buildroot that shows the tests 
> passing on various hardware configurations.

nice.

the infra in libc-test is not amazing for continous tests.
when i get to it i will try to improve it.

> 
> Looking forward to your guidance,
> Ryan Gardner

Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

[Szabolcs Nagy]

* Gardner, Ryan P <ryan.p.gardner@boeing.com> [2024-11-07 04:39:59 +0000]:

> Szabolcs Nagy <nsz@...t70.net> writes:
> 
> > thanks
> > patches look good. but it will take a week or two
> > for me to be able to test and apply them.
> 
> Thank you for the quick response. We appreciate your timeline for testing and applying the patches.

applied the patches and added you to authors
(which is not very formal, more for acknowledgement)

thanks.

> 
> Our plan is to submit tests for the majority of functions that currently lack coverage within libc-test. 
> We already have a significant number of tests written and ready for submission. To avoid overwhelming
> you with patches, do you have a preferred cadence for the submission of these tests?
> 
> Additionally, we are planning to setup runtime tests in collaboration with the Buildroot community. This
> would facilitate testing across various embedded configurations and processor architectures. Once up 
> and running, when we submit new test patches we can reference a fork of Buildroot that shows the tests 
> passing on various hardware configurations.
> 
> Looking forward to your guidance,
> Ryan Gardner

[musl] RE: [EXTERNAL] Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

[Ward, Ryan C]

*Szabolcs Nagy <nsz@...t70.net> writes:
> Gardner, Ryan P <ryan.p.gardner@boeing.com> [2024-11-07 04:39:59 +0000]:
> > Szabolcs Nagy <nsz@...t70.net> writes:
> >  
> > > thanks
> > > patches look good. but it will take a week or two for me to be able 
> > > to test and apply them.
> > 
> > Thank you for the quick response. We appreciate your timeline for testing and applying the patches.
> > 
> > Our plan is to submit tests for the majority of functions that currently lack coverage within libc-test. 
> > We already have a significant number of tests written and ready for 
> > submission. To avoid overwhelming you with patches, do you have a preferred cadence for the submission of these tests?
> > 
>
> over the next month im only in front of a computer sporadically
>
> but code style or quality of tests are less strict than for libc, so i dont expect many iterations. just post the patches so when i can take a look i can comment or apply them.
>
> btw if there is a repo with a branch i can cherrypick from that is a bit easier for me than handling mails. then i think you dont even have to post "obvious" patches to the list. i can pick them from the branch.

Hi, we have now pushed some scripts we have used during development to
to our external staging branch, located here:

https://github.com/Boeing/libc-test/tree/boeing-staging-next-libc-test/rfc/rfc_add_testing_scripts 

This email serves as an RFC for adding some test scripting/infrastructure, with
an overview provided:

run_tests:
A simple shell script that was written to assist with continuous testing, 
both locally and a CI/CD pipeline.

The script serves to provide a more verbose output of both passed and failed
tests, and assumes that musl has been installed in the default directory of
`/usr/local/musl`.

The `run_tests` script can be used with the `-r` flag to generate test coverage
reports of libc-test on the musl shared object. However, this option is more involved, 
and requires the user to have at least clang and llvm 19 installed on the host system. 
It also requires that `musl` has been build, installed and instrumented with the 
`-fprofile-instr-generate` and `-fcoverage-mapping` flags.

Note that some `*printf_chk()` functions will need to be implemented (as stubs)
for musl to compile. An example patch has been added, outlining the necessary
functions that need to be implemented.

stress_test:
Another simple shell script to run newly added multiple times to catch any 
missed edge cases, particularly in the context of timing/synchronisation.

> > Additionally, we are planning to setup runtime tests in collaboration 
> > with the Buildroot community. This would facilitate testing across 
> > various embedded configurations and processor architectures. Once up 
> > and running, when we submit new test patches we can reference a fork of Buildroot that shows the tests passing on various hardware configurations.
>
> nice.
>
> the infra in libc-test is not amazing for continous tests.
> when i get to it i will try to improve it.

Kind regards,

Ryan W

Re: [musl] RE: [EXTERNAL] Re: [musl] [PATCH 1/3 libc-test] functional:time:clock_nanosleep test

[Szabolcs Nagy]

* Ward, Ryan C <ryan.c.ward3@boeing.com> [2024-12-04 01:19:34 +0000]:

> *Szabolcs Nagy <nsz@...t70.net> writes:
> > Gardner, Ryan P <ryan.p.gardner@boeing.com> [2024-11-07 04:39:59 +0000]:
> > > Szabolcs Nagy <nsz@...t70.net> writes:
> > >  
> > > > thanks
> > > > patches look good. but it will take a week or two for me to be able 
> > > > to test and apply them.
> > > 
> > > Thank you for the quick response. We appreciate your timeline for testing and applying the patches.
> > > 
> > > Our plan is to submit tests for the majority of functions that currently lack coverage within libc-test. 
> > > We already have a significant number of tests written and ready for 
> > > submission. To avoid overwhelming you with patches, do you have a preferred cadence for the submission of these tests?
> > > 
> >
> > over the next month im only in front of a computer sporadically
> >
> > but code style or quality of tests are less strict than for libc, so i dont expect many iterations. just post the patches so when i can take a look i can comment or apply them.
> >
> > btw if there is a repo with a branch i can cherrypick from that is a bit easier for me than handling mails. then i think you dont even have to post "obvious" patches to the list. i can pick them from the branch.
> 
> Hi, we have now pushed some scripts we have used during development to
> to our external staging branch, located here:
> 
> https://github.com/Boeing/libc-test/tree/boeing-staging-next-libc-test/rfc/rfc_add_testing_scripts 
> 
> This email serves as an RFC for adding some test scripting/infrastructure, with
> an overview provided:

thanks for doing this.

fyi i'm travelling and my laptop got stolen.

will be able to recover. but it will take a while.
the integrity of the repo is not in danger.

> 
> run_tests:
> A simple shell script that was written to assist with continuous testing, 
> both locally and a CI/CD pipeline.
> 
> The script serves to provide a more verbose output of both passed and failed
> tests, and assumes that musl has been installed in the default directory of
> `/usr/local/musl`.
> 
> The `run_tests` script can be used with the `-r` flag to generate test coverage
> reports of libc-test on the musl shared object. However, this option is more involved, 
> and requires the user to have at least clang and llvm 19 installed on the host system. 
> It also requires that `musl` has been build, installed and instrumented with the 
> `-fprofile-instr-generate` and `-fcoverage-mapping` flags.
> 
> Note that some `*printf_chk()` functions will need to be implemented (as stubs)
> for musl to compile. An example patch has been added, outlining the necessary
> functions that need to be implemented.
> 
> stress_test:
> Another simple shell script to run newly added multiple times to catch any 
> missed edge cases, particularly in the context of timing/synchronisation.
> 
> > > Additionally, we are planning to setup runtime tests in collaboration 
> > > with the Buildroot community. This would facilitate testing across 
> > > various embedded configurations and processor architectures. Once up 
> > > and running, when we submit new test patches we can reference a fork of Buildroot that shows the tests passing on various hardware configurations.
> >
> > nice.
> >
> > the infra in libc-test is not amazing for continous tests.
> > when i get to it i will try to improve it.
> 
> Kind regards,
> 
> Ryan W