* Missing tests @ 2014-05-14 13:05 Rich Felker 2014-05-14 14:30 ` Szabolcs Nagy 0 siblings, 1 reply; 3+ messages in thread From: Rich Felker @ 2014-05-14 13:05 UTC (permalink / raw) To: musl Here are some things that don't seem to be tested that would be nice to get added to the test suite (http://nsz.repo.hu/git/?p=libc-test) - Interpretation of POSIX TZ strings - Daylight time transition tests with zoneinfo and POSIX zones - Lots of corner cases for strftime and strptime - Regression test for recent pthread_once race (hard to catch) - Sanity-checking ucontext_t layout via signal handlers - Alternate signal stacks - Basic sanity checks for most syscalls - Checks for various subtle stdio behavior: - File position on exit - Interactions of ungetc and file position - Append mode file positions It might make sense to make a new tree separate from functional for tests that don't really check the functionality of the interface but just whether it's present and "doing something". Some existing tests might be candidates for moving there too..? Rich P.S. I started writing this email quite a while back when I had connectivity problems and never got around to finishing/sending it, so some of it is motivated by discussions that were going on at the time. :-) ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Missing tests 2014-05-14 13:05 Missing tests Rich Felker @ 2014-05-14 14:30 ` Szabolcs Nagy 2014-05-20 19:49 ` Rich Felker 0 siblings, 1 reply; 3+ messages in thread From: Szabolcs Nagy @ 2014-05-14 14:30 UTC (permalink / raw) To: musl * Rich Felker <dalias@libc.org> [2014-05-14 09:05:30 -0400]: > Here are some things that don't seem to be tested that would be nice > to get added to the test suite (http://nsz.repo.hu/git/?p=libc-test) > > - Interpretation of POSIX TZ strings > - Daylight time transition tests with zoneinfo and POSIX zones > - Lots of corner cases for strftime and strptime > - Regression test for recent pthread_once race (hard to catch) > - Sanity-checking ucontext_t layout via signal handlers > - Alternate signal stacks i have a (regression) test for sigaltstack > - Basic sanity checks for most syscalls this can be tricky > It might make sense to make a new tree separate from functional for > tests that don't really check the functionality of the interface but > just whether it's present and "doing something". Some existing tests > might be candidates for moving there too..? the code organization is not optimal we can add a directory for trivial tests the difficult part is when the test needs special environment (eg syscall sanity checks or getaddrinfo): users/groups, root-priv, fs-access, /etc/ changes,... we can collect them in separate dirs and run only when in a completely isolated environment (eg qemu) linux specific tests should be separate too another todo item is to improve cross compilation (running cross compiled tests does not really work now because it is done from make) (the dso tests are a bit broken too because they use relative path so only work when started from a specific dir, maybe such test should be started from an .sh script) ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Missing tests 2014-05-14 14:30 ` Szabolcs Nagy @ 2014-05-20 19:49 ` Rich Felker 0 siblings, 0 replies; 3+ messages in thread From: Rich Felker @ 2014-05-20 19:49 UTC (permalink / raw) To: musl On Wed, May 14, 2014 at 04:30:46PM +0200, Szabolcs Nagy wrote: > > It might make sense to make a new tree separate from functional for > > tests that don't really check the functionality of the interface but > > just whether it's present and "doing something". Some existing tests > > might be candidates for moving there too..? > > the code organization is not optimal > > we can add a directory for trivial tests > > the difficult part is when the test needs special environment > (eg syscall sanity checks or getaddrinfo): users/groups, > root-priv, fs-access, /etc/ changes,... > we can collect them in separate dirs and run only > when in a completely isolated environment (eg qemu) For getaddrinfo, I want a way to override the config via an environment variable or similar, but I'm not sure what the right way to do it would be. If we come up with a working solution, then the tests could provide a fake DNS, and even do things like generate malformed responses, to test the resolver. Some of the others might be possible to test with user namespaces/containers without needing root, but that would need to be an option since it doesn't work on non-Linux or old-Linux systems. > linux specific tests should be separate too And musl-specific ones, if we have any (ones which test for behaviors that musl documents but which are not required by the standards, like AS-safety of snprintf, dprintf, etc.). > another todo item is to improve cross compilation > (running cross compiled tests does not really work > now because it is done from make) > (the dso tests are a bit broken too because they use > relative path so only work when started from a specific > dir, maybe such test should be started from an .sh script) Yes, it would be nice if running the tests could be separated from building them, but without sacrificing the declarative makefile which is ideal for native testing. Rich ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2014-05-20 19:49 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-05-14 13:05 Missing tests Rich Felker 2014-05-14 14:30 ` Szabolcs Nagy 2014-05-20 19:49 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).