Re: Weekly reports: A

[Rich Felker <dalias@aerifal.cx>]

On Tue, May 24, 2011 at 05:00:29AM +0400, Solar Designer wrote:
> Luka, Rich -
> 
> On Mon, May 23, 2011 at 07:42:38PM +0200, Luka Mar??eti?? wrote:
> > average, I expect to complete one task Rich described at 
> > http://openwall.info/wiki/musl/unit-tests?s=libc per each report. I'll 
> > start with nr. 1 (though there is nr. 0), as it allows me to correct and 
> > complete the proof-of-skill test I've written earlier.
> 
> (There are 13 test categories currently listed on the wiki page.)
> 
> Sounds fine to me.  I assume that you'll also proceed to wrap those
> tests into a framework once you have a few initial tests implemented.

Yes, I'd like him to get some more tests working first though. A good
design for the framework should become more apparent when you have
tests to try integrating.

> For "String operations testing" with "giant buffers (more than 2gb/4gb,
> only possible on 64-bit machines)" (part of task nr. 1 that you intend
> to work on now), you may consider having this test run on systems that
> don't have this much virtual memory (but are 64-bit capable).  You may
> achieve this by mmap()'ing the same pages many times.  When I needed a
> large continuous pseudo-allocation like that to explore/exploit some
> Linux kernel issues, I was able to allocate something like 190 GB (if I
> recall correctly) on an Alpha with 128 MB RAM (that was in 1999, before
> we got x86-64).  (The kernel would spend 25 minutes parsing that data.)

Another option is using a giant file. Even if you don't want to take
that approach, you could start with a giant sparse file to mmap in
order to allocate the virtual address space, then mmap over top of it
with the MAP_FIXED option to achieve the above.

> Oh, it's also useful to test buffer sizes close to 2 GB and slightly
> over 2 GB on 32-bit systems that are capable of such allocations.

musl's malloc refuses to allocate >2gb on 32-bit systems to avoid the
ptrdiff_t overflow issue, but mmap could still potentially create such
large objects.

> For "low and high byte content", I suggest that you include ability to
> test all byte values (for non-wide chars).  glibc and many other libc's
> include implementations of string functions that use adds/bitmasks;
> these might contain bugs that only show up with specific byte values in
> specific character positions when the libc is built for specific CPUs.

I agree. I don't believe any such issues affect the current C
implementations in musl, but it would be nice to have the tests in
place in case anyone wants to add arch-specific asm versions.

> These are just some suggestions, which Rich might override. ;-)

They sound very good to me.

Rich