From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/114 Path: news.gmane.org!not-for-mail From: Szabolcs Nagy Newsgroups: gmane.linux.lib.musl.general Subject: Re: Re: Daily reports: Wednesday Date: Thu, 7 Jul 2011 18:18:32 +0200 Message-ID: <20110707161832.GQ27634@port70.net> References: <4E125DBC.9090809@gmail.com> <4E131E8F.9@gmail.com> <4E14C55E.6030808@gmail.com> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1310055532 21969 80.91.229.12 (7 Jul 2011 16:18:52 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 7 Jul 2011 16:18:52 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-198-gllmg-musl=m.gmane.org@lists.openwall.com Thu Jul 07 18:18:48 2011 Return-path: Envelope-to: gllmg-musl@lo.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by lo.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1QerHp-0000dy-HZ for gllmg-musl@lo.gmane.org; Thu, 07 Jul 2011 18:18:45 +0200 Original-Received: (qmail 5673 invoked by uid 550); 7 Jul 2011 16:18:44 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 5658 invoked from network); 7 Jul 2011 16:18:44 -0000 Content-Disposition: inline In-Reply-To: <4E14C55E.6030808@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Xref: news.gmane.org gmane.linux.lib.musl.general:114 Archived-At: * Luka Mar??eti?? [2011-07-06 22:28:14 +0200]: > As for the future, I'm planing on making the cluts.c framework, and > this is what I intend for it to do: > > * find 'test collections' using dirent.h (haven't used it yet, but > shouldn't be hard to learn I guess) > * fork off for each test collection and execl() each one > * simple statistics based on the status provided by wait() > my opinion: don't overengineer the test system (i have ideas about it, i think i'll implement them soon, but i'll be offline for a week) as a first round it's enough to make it work from a makefile/shell script (ie. writing a testsuit tool in c with flags etc is not that important) a simple solution is: test collections are .c files in directories each directory makes one executable (several test_foo.c with some exported test_foo functions and one main, the main can be even generated if test_* convention is used but at first hand writing it is probably simpler) there is a shell script/makefile in the root that builds and runs the tests in all directory and gives a report at the end about the number of failures simplicity is important so that it is easy to contribute tests or fix the system later when there are more tests extra features can come later: > * advanced statistics(via what will be common/ipc.c): shared memory > with two integers for each 'test collection' which will indicate: > a) number of tests that have begun executing (incremented before > each test starts by the child) > b) number of successful tests > The pointer to shared memory will be passed as an argument to > the child (test collection). From a) the framework will be able to > know which test number has crashed the collection. If the collection > returns, the framework will be able to calculate the success rate by > dividing b) with a). > * a few features which the user can invoke via some switches (using > standard tokenizing functions - need to study them first): > -v --verbose without which collections will not print anything > themselves > -t --test [,,...] specific tests to execute > -x --exclude [,,...] exceptionally excluded tests > -h --help etc > at some point (but probably not right now) it would be nice if slow tests can be marked with some macro oslt, by the test writer and it should be easy to avoid those i'm not sure about -t and -x if long and descriptive testnames are used then a regex filter is probably easier to use than -t -x (but we don't know this yet so it should be decided when there are many tests ready) > Comments and suggestions are very welcome. Especially any advice > about includes and .h files is welcome, I am new to structuring > files in C. if you are new to it then i really recommend to do the absolute simplest/minimal thing you can imagine here, then it will be easy to fix later.. of course ymmv