From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/5685 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Building a solid musl automated-testing framework Date: Thu, 31 Jul 2014 17:47:16 -0400 Message-ID: <20140731214716.GA27647@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1406843257 3220 80.91.229.3 (31 Jul 2014 21:47:37 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 31 Jul 2014 21:47:37 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-5690-gllmg-musl=m.gmane.org@lists.openwall.com Thu Jul 31 23:47:30 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1XCyC9-0004Xa-K5 for gllmg-musl@plane.gmane.org; Thu, 31 Jul 2014 23:47:29 +0200 Original-Received: (qmail 9816 invoked by uid 550); 31 Jul 2014 21:47:28 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 9805 invoked from network); 31 Jul 2014 21:47:28 -0000 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:5685 Archived-At: I'd like to figure out how to setup the openadk test framework, or adapt things from it, for automated testing all musl ports. The repo is here: http://www.openadk.org/cgi-bin/gitweb.cgi?p=adk-test-framework.git There's a lot of stuff hard-coded for the openadk toolchains, whereas I'd like to be able to use it with the musl-cross toolchains which are more canonical. The scripts also seem to be incompatible with busybox (using GNU features in something for making the initramfs, probably cpio?). And by default it tries to test musl 1.0.1 and doesn't have an obvious way to test from git. The idea I have in mind for using it as a basis for automated testing would be: 1. Have a Makefile and a subdirectory full of cross compiler trees. The set of archs could be based purely on the set of cross compilers, computed by the Makefile based on directory contents. 2. Have the Makefile clone/pull one musl tree from upstream git and then clone/pull an additional copy of the tree per arch/subarch (with the set of archs determined as in point 1). 3. Build each musl (via recursive make, i.e. make calling configure and make in each musl clone) and install them into the cross compiler include/lib dirs for their corresponding cross compilers. 4. Do like points 2 and 3, but for libc-test repo. 5. Use the kernel, initrd, and qemu stuff from the adk-test-framework to build initrd images and run them. The idea of using a Makefile (aside from declarative being The Right Way to do anything like this :) is to make it easy to make a single fix and re-run the test without rebuilding everything. Of course rebuilding (via a "make clean" or similar) should aof coursebe an option and what's usually used. Does any of this make sense? Or is it simpler to get this kind of automated testing just making a few tweaks to the existing scripts? It would be ideal if the whole thing would work with dumping the test repo on a system not necessarily setup for running it (e.g. some kind of build farm or 'cloud computing' resource) and just produce the results that could be read offline. Rich