From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/3676 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: Request for volunteers Date: Wed, 17 Jul 2013 11:41:29 -0400 Message-ID: <20130717154129.GE12469@brightrain.aerifal.cx> References: <20130630055202.GA16580@brightrain.aerifal.cx> <20130630104834.GP15323@port70.net> <20130701035117.GX29800@brightrain.aerifal.cx> <20130701205856.GS15323@port70.net> <20130701235954.GD29800@brightrain.aerifal.cx> <20130702021937.GT15323@port70.net> <20130702074920.GF29800@brightrain.aerifal.cx> <20130716162030.GQ15323@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1374075703 4608 80.91.229.3 (17 Jul 2013 15:41:43 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 17 Jul 2013 15:41:43 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-3680-gllmg-musl=m.gmane.org@lists.openwall.com Wed Jul 17 17:41:45 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1UzTrL-0004eE-CN for gllmg-musl@plane.gmane.org; Wed, 17 Jul 2013 17:41:43 +0200 Original-Received: (qmail 17753 invoked by uid 550); 17 Jul 2013 15:41:42 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 17745 invoked from network); 17 Jul 2013 15:41:42 -0000 Content-Disposition: inline In-Reply-To: <20130716162030.GQ15323@port70.net> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:3676 Archived-At: On Tue, Jul 16, 2013 at 06:20:31PM +0200, Szabolcs Nagy wrote: > * Rich Felker [2013-07-02 03:49:20 -0400]: > > What about a mix? Have the makefile include another makefile fragment > > with a rule to generate that fragment, where the fragment is generated > > from comments in the source files. Then you have full dependency > > tracking via make, and self-contained tests. > > i wrote some tests but the build system became a bit nasty > i attached the current layout with most of the test cases > removed so someone can take a look and/or propose a better > buildsystem before i do too much work in the wrong direction If you really want to do multiple makefiles, what about at least setting up the top-level makefile so it invokes them via dependencies rather than a shell for-loop? > each directory has separate makefile because they work > differently > > functional/ and regression/ tests have the same makefile, > they set up a lot of make variables for each .c file in > the directory, the variables and rules can be overridden > by a similarly named .mk file > (this seems to be more reliable than target specific vars) > > (now it builds both static and dynamic linked binaries > this can be changed) It's probably useful. We had plenty of bugs that only showed up one way or the other, but it may be useful just to test the cases where we know or expect it matters (purely in the interest of build time and run time). > i use the srcdir variable so it is possible to build > the binaries into a different directory (so a single > source tree can be used for glibc and musl test binaries) > i'm not sure how useful is that (one could use several > git repos as well) If it's easy to do, I like it. It makes it easy to try local changes on both without committing them to a repo. > another approach would be one central makefile that > collects all the sources and then you have to build > tests from the central place > (but i thought that sometimes you just want to run > a subset of the tests and that's easier with the > makefile per dir approach, I would like this better, but I'm happy to have whatever works. IMO it's not too bad to support building subsets with a single makefile. You just have variables containing the names of all tests in a particular subset and rules that depend on just those tests. One thing I also just thought of is that you could have separate REPORT files for each test which are concatenated to the final REPORT file. This makes it possible to run the tests in parallel. In general, I think the more declarative/functional and less procedural you make a makefile, the simpler it is and the better it works. > another issue is dlopen > and ldso tests need the .so binary at the right path > at runtime so you cannot run the tests from arbitrary > directory) Perhaps the makefile could pass the directory containing the test as an argument to it for these tests so they could chdir to their own location as part of the test? > yet another approach would be to use a simple makefile > with explicit rules without fancy gnu make tricks > but then the makefile needs to be edited whenever a > new test is added I like the current approach where you don't have to edit the makefile. :-) > i'm not sure what's the best way to handle common/ > code in case of decentralized makefiles, now i > collected them into a separate directory that is > built into a 'libtest.a' that is linked to all > tests so you have to build common/ first That's why I like unified makefiles. > i haven't yet done proper collection of the reports > and i'll need some tool to run the test cases: > i don't know how to report the signal name or number > (portably) from sh when a test is killed by a signal Instead of using the shell, run it from your own program that gets the exit status with waitpid and passes that to strsignal. > (the shell prints segfault etc to its stderr which may > be used) and i don't know how to kill the test reliably > after a timeout > > i hope this makes sense Yes. Hope this review is helpful. Again, this is your project and I'm very grateful that you're doing it, so I don't want to impose my opinions on how to do stuff, especially if it hinders your ability to get things done. Thanks and best wishes, Rich