Re: Missing tests

[Rich Felker <dalias@libc.org>]

On Wed, May 14, 2014 at 04:30:46PM +0200, Szabolcs Nagy wrote:
> > It might make sense to make a new tree separate from functional for
> > tests that don't really check the functionality of the interface but
> > just whether it's present and "doing something". Some existing tests
> > might be candidates for moving there too..?
> 
> the code organization is not optimal
> 
> we can add a directory for trivial tests
> 
> the difficult part is when the test needs special environment
> (eg syscall sanity checks or getaddrinfo): users/groups,
> root-priv, fs-access, /etc/ changes,...
> we can collect them in separate dirs and run only
> when in a completely isolated environment (eg qemu)

For getaddrinfo, I want a way to override the config via an
environment variable or similar, but I'm not sure what the right way
to do it would be. If we come up with a working solution, then the
tests could provide a fake DNS, and even do things like generate
malformed responses, to test the resolver.

Some of the others might be possible to test with user
namespaces/containers without needing root, but that would need to be
an option since it doesn't work on non-Linux or old-Linux systems.

> linux specific tests should be separate too

And musl-specific ones, if we have any (ones which test for behaviors
that musl documents but which are not required by the standards, like
AS-safety of snprintf, dprintf, etc.).

> another todo item is to improve cross compilation
> (running cross compiled tests does not really work
> now because it is done from make)
> (the dso tests are a bit broken too because they use
> relative path so only work when started from a specific
> dir, maybe such test should be started from an .sh script)

Yes, it would be nice if running the tests could be separated from
building them, but without sacrificing the declarative makefile which
is ideal for native testing.

Rich