From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/5709 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Building a solid musl automated-testing framework Date: Mon, 4 Aug 2014 10:56:26 -0400 Message-ID: <20140804145626.GW1674@brightrain.aerifal.cx> References: <20140731214716.GA27647@brightrain.aerifal.cx> <20140804121855.GO32743@waldemar-brodkorb.de> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1407164206 4137 80.91.229.3 (4 Aug 2014 14:56:46 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 4 Aug 2014 14:56:46 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-5714-gllmg-musl=m.gmane.org@lists.openwall.com Mon Aug 04 16:56:40 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1XEJgl-0001do-5v for gllmg-musl@plane.gmane.org; Mon, 04 Aug 2014 16:56:39 +0200 Original-Received: (qmail 7851 invoked by uid 550); 4 Aug 2014 14:56:38 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 7843 invoked from network); 4 Aug 2014 14:56:38 -0000 Content-Disposition: inline In-Reply-To: <20140804121855.GO32743@waldemar-brodkorb.de> User-Agent: Mutt/1.5.21 (2010-09-15) Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:5709 Archived-At: On Mon, Aug 04, 2014 at 02:18:55PM +0200, Waldemar Brodkorb wrote: > Hi, > Rich Felker wrote, > > > I'd like to figure out how to setup the openadk test framework, or > > adapt things from it, for automated testing all musl ports. The repo > > is here: > > > > http://www.openadk.org/cgi-bin/gitweb.cgi?p=adk-test-framework.git > > > > There's a lot of stuff hard-coded for the openadk toolchains, whereas > > I'd like to be able to use it with the musl-cross toolchains which are > > more canonical. The scripts also seem to be incompatible with busybox > > (using GNU features in something for making the initramfs, probably > > cpio?). And by default it tries to test musl 1.0.1 and doesn't have an > > obvious way to test from git. > > I have copied the adk-test-framework logic to a new project. > http://www.openadk.org/cgi-bin/gitweb.cgi?p=embedded-test.git;a=summary > > It is more general and supports buildroot instead of openadk to make > tests of toolchain building and runtime testing for glibc, uclibc, > uclibc-ng and musl. The big issue with using it directly is that we're not trying to test whole toolchains, just musl, and testing toolchains instead of just musl turns a 5 minute build (that's easy to run and re-run whenever you want to test) into an hours-long process. > The benefit is, you do not need any ready to go kernel and userland, > because everything got compiled from scratch. So any fixes to > kernel config or userland can be done. > > If there is an interest, I can add support for using an existing > toolchain (like musl-cross) to build a kernel and userland to do > the runtime testing. I'd still much prefer to have the kernel built outside of the test system, since it's not a component to be tested and in no way uses or depends on the components to be tested. The situation is similar for busybox, which is just used as a shell for executing the tests, not as a component to be tested itself, but here there's also a consideration of not introducing regressions in the shell that's controlling the tests. I'd like to be using a "known good" shell and have only the tests themselves linking against the new musl being tested. Rich