From: Waldemar Brodkorb <wbx@openadk.org>
To: musl@lists.openwall.com
Subject: Re: Building a solid musl automated-testing framework
Date: Wed, 6 Aug 2014 09:57:11 +0200 [thread overview]
Message-ID: <20140806075711.GQ32743@waldemar-brodkorb.de> (raw)
In-Reply-To: <20140804145626.GW1674@brightrain.aerifal.cx>
Hi,
Rich Felker wrote,
> On Mon, Aug 04, 2014 at 02:18:55PM +0200, Waldemar Brodkorb wrote:
> > Hi,
> > Rich Felker wrote,
> >
> > > I'd like to figure out how to setup the openadk test framework, or
> > > adapt things from it, for automated testing all musl ports. The repo
> > > is here:
> > >
> > > http://www.openadk.org/cgi-bin/gitweb.cgi?p=adk-test-framework.git
> > >
> > > There's a lot of stuff hard-coded for the openadk toolchains, whereas
> > > I'd like to be able to use it with the musl-cross toolchains which are
> > > more canonical. The scripts also seem to be incompatible with busybox
> > > (using GNU features in something for making the initramfs, probably
> > > cpio?). And by default it tries to test musl 1.0.1 and doesn't have an
> > > obvious way to test from git.
> >
> > I have copied the adk-test-framework logic to a new project.
> > http://www.openadk.org/cgi-bin/gitweb.cgi?p=embedded-test.git;a=summary
> >
> > It is more general and supports buildroot instead of openadk to make
> > tests of toolchain building and runtime testing for glibc, uclibc,
> > uclibc-ng and musl.
>
> The big issue with using it directly is that we're not trying to test
> whole toolchains, just musl, and testing toolchains instead of just
> musl turns a 5 minute build (that's easy to run and re-run whenever
> you want to test) into an hours-long process.
Only the first run will create a toolchain. On the second run
without -c, only the C library will be recompiled and installed.
> > The benefit is, you do not need any ready to go kernel and userland,
> > because everything got compiled from scratch. So any fixes to
> > kernel config or userland can be done.
> >
> > If there is an interest, I can add support for using an existing
> > toolchain (like musl-cross) to build a kernel and userland to do
> > the runtime testing.
>
> I'd still much prefer to have the kernel built outside of the test
> system, since it's not a component to be tested and in no way uses or
> depends on the components to be tested.
I do not agree here. For example the latest kernel includes
File-private POSIX locks.
https://lwn.net/Articles/586904/
If you want to add the needed bits and bytes to the C library it
would be nice to test it. Therefore you need the latest kernel.
> The situation is similar for
> busybox, which is just used as a shell for executing the tests, not as
> a component to be tested itself, but here there's also a consideration
> of not introducing regressions in the shell that's controlling the
> tests. I'd like to be using a "known good" shell and have only the
> tests themselves linking against the new musl being tested.
Recently I discovered a bug in uClibc in the machine dependent pipe
implementation for sparc by using mksh and its test suite.
So running application specific test suites for mksh/busybox might
discover C library bugs. So I want to support it.
Do you want to use a glibc basesystem to run the musl tests with
musl C library as in adk-test-framework or do you want to simply run
the test on a musl based system?
best regards
Waldemar
prev parent reply other threads:[~2014-08-06 7:57 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-07-31 21:47 Rich Felker
2014-08-01 6:32 ` u-igbb
2014-08-04 5:10 ` Bobby Bingham
2014-08-04 5:30 ` Szabolcs Nagy
2014-08-04 12:18 ` Waldemar Brodkorb
2014-08-04 14:56 ` Rich Felker
2014-08-06 7:57 ` Waldemar Brodkorb [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140806075711.GQ32743@waldemar-brodkorb.de \
--to=wbx@openadk.org \
--cc=musl@lists.openwall.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).