Hej, I started running the context test suite as a regression test to verify that my local changes don't introduce any errors. I got LOTS of errors. While I may have introduced som errors, I really couldn't have introduced THAT many errors. Testing was also somewhat erratic. Sometimes things worked, sometimes not. So I then ran the test suite using the unmodified standalone context, with the original binary. I ran the test suite eight times, and then compared each run with the previous, so there are seven comparisons for each test. For each test, I then counted the number that were different. Ideally, there should either be no differences, or, if the test uses random numbers, seven differences. But there are a number of tests that generate between one and six errors. In total, 324 of the tests do not consistently generate identical output. I have put an archive of context diffs less than 100k is size here: http://www.sdu.se/pub/luatex/ There is also the script that I use to compare two runs on the test suite. Note that I remove a few lines that always differ. In addition I remove some XML lines. Each test is run like this context --silent --result=/path/to/output/01/cld/functions-001.pdf functions-001.tex I have identified a few different errors, and will describe them in separate posts. /Lars ------------------------------------------------------------------- Lars Hamrén WWW...: www.sdu.se Vadmöllan 211 Mobile: +46(705)189090 S-225 94 Lund Sweden
Hi, Thanks to Hans, most tests now run deterministically and the generated PDFs are identical down to the last bit. Remaining are - some tests that depend on the key order of hash (which varies from run to run) - some test that depend on when, or for how long, the test executes. - a bug. Difference is invisible, but PDF differs Also, some tests depend on non-standard, possibly commercial, fonts. To run the test, these options to context are suggested: --nocompression --randomseed=471174 --nodates --trailerid=1 Fixes are available the the very latest beta, and in the latest test archive. I have attached my "skip list" of files not to include in the test. /Lars ------------------------------------------------------------------- Lars Hamrén WWW...: www.sdu.se Vadmöllan 211 Mobile: +46(705)189090 S-225 94 Lund Sweden
participants (2)
-
Lars Hamren
-
Lars Hamren