Re: test suite in the garden

["Patrick Gundlach" <patrick@gundla.ch>]

>> It is perhaps a lot of work included, but extending the
>> contextgarden.net in such a way that users could provide test cases
>> which would be typeset with different ConTeXt versions and PNGs
>> compared afterwards ... could make it easier to discover any broken
>> functionality.

It comes down to:

* different context versions needed
* different test documents needed

1) user can select any combination of the above
2) result (one page/png, more pages pdf) can be viewed or downloaded
3) feedback

or perhaps (as I can read between your lines)

a pdf with even number of pages:
left page: original, right page the selcted version. 

So we need test documents and after that context can mix in the pages
from the known good document and the selecte version. 

So you think that users actually download these comparisons?

> Oh, another quest for our magic Patrick! ;-)

Don't call me magic. I am currently trying to understand the tftopl,
pltotf, vftovp and vptovf programs in detail. And I am so miserable at
that; all kinds of optimizations that makes the code unreadable. 

> Let me extend the suggestion, for we miss a ConTeXt test suite for a
> long time now:
>
> - users send test cases
> - test cases get typeset with the actual ConTeXt and converted to PNG
>   (like inline samples)
> - user "votes" if it looks right, if not adds comment
> - test case gets saved, including PNG, ConTeXt version and vote/comments
> - at next update, all test cases get typeset again, and users can
>   "vote" if it looks still ok or at least the same (perhaps it would
>   be  possible to check automatically if the bitmaps are exactly
>   identical)

Again, I doubt that users will actually do these kind of things. I
could provide some interface for documents connected to version
numbers, so you could download a set of .tex files (or one big tex
file) and the related pdf file, but we need to collect good examples.
And I think that testing is very hard: there are so many different
things in ConTeXt that can be tested, so the pdf would result in a few
hundred pages. 

Patrick
-- 
ConTeXt wiki and more: http://contextgarden.net