|
| | |
Nearly all tests for Xalan are independent Java classes built
into testxsl.jar that
can be run either individually on the command line, programmatically
from your application or our handy Ant build.xml file, or in batches from
XSLTestHarness.
There really isn't any magic to them: you can just set your classpath and
execute java.exe to run them; some Tests and Testlets currently provide defaults
for their inputs, so you can run them without any setup at all.
However we have provided a couple of more
convenient ways to run the most common tests:
 | If you need to debug into the tests themselves,
you may want to use the debug*.bat files |
Of course, first Build a fresh copy of testxsl.jar.
cd xml-xalan\test
You can either: use the Ant build.xml script; run a convenience batch file; or execute java.exe yourself.
build conf [Ant-prefixed options]
(runs StylesheetTestletDriver over tests\conf test tree using the default StylesheetTestlet)
build perf [Ant-prefixed options]
(runs StylesheetTestletDriver over tests\perf test tree using the default PerformanceTestlet)
build api -DtestClass=TRAXAPITestClassName [Ant-prefixed options]
(runs TRAX interface tests with Xalan-J 2.x
Alternately: some convenience batch files are provided for the most common
tests - these simply turn around and call build.bat/.sh for you.
(Namely conf.bat, perf.bat, and the like)
Alternately: Run java.exe and the desired test class yourself:
| Running tests with alternate JAXP parsers: all org.apache.qetest.trax.*
tests can be run with Xalan-J 2.x and any JAXP 1.1 compatible parser, like
crimson.jar. Be sure to manually set the appropriate system properties to use
your parser instead of xerces.jar, which is the default for Xalan-J. Tests will
also run on any JAXP 1.1 compatible xslt processor, namely either our default
xalan one or our new xsltc one. |
|
 | | | | How-to: View Test Results | | | | |
| | |
Most tests both send basic results to System.out, as well as
writing a full set of output results to their logFile, as
set from a .properties file or the command line. Generally the
output results file is easier to deal with. The basic format is
fairly simple and you can certainly read it as-is. Also, many tests
send only summary results to the console, but full output to the results file.
To 'pretty-print' results or produce reports, please use the
viewResults.xsl stylesheet and associated batch/shell files, like so:
cd \xml-xalan\test
build conf -DoptionName=optionValue -Dqetest.logFile results/MyResults.xml
viewResults.bat results/MyResults.xml results/MyPrettyResults.html [options]
These options are passed to Xalan's command line to transform the
xml formatted output results into a nicer-looking html file. The most
common option would be -param loggingLevel 99 to get
more output shown in the results (higher numbers up to 99 show more details,
lower numbers, down to 0, show fewer details).
Alternatively, the tableResults.xsl stylesheet can be used to pretty-print
results from the conformance test for multiple TRAX flavours:
cd \xml-xalan\test
build alltest.conf
set RESULTSCANNER=tableResults.xsl
viewResults.bat results-alltest/conf/sax/results.xml results.html [options]
This will generate a pretty-printed HTML table in results.html, listing all
flavours as well as the results of individual test cases for each category.
The options can be passed to tableResults.xsl to generate conformance reports
on previously run tests, or to compare two runs of the conformance test suite.
Possible options that can be passed to tableResults.xsl:
-
-param resultsDir /path/to/result/results-alltest/conf
Passing the path to the results-alltest/conf directory generates
a report based on the results in that directory. Defaults to ./results-alltest/conf
-
-param compareAgainst /path/to/other/results-alltest/conf
Passing the path to anothe results-alltest/conf directory compares
that result against the result pointed to by resultsDir.
-
-param resultsDir results-alltest.xsltc/conf
Generates a report on the results of an xsltc test (with the alltest.conf.xsltc) target.
Future work includes greatly updated results analysis stylesheets.
See FailScanner.xsl and PerfScanner.xsl for ideas. An important design
principle in the tests is that at runtime the tests merely output
whatever data they can as results are generated; afterwards, we then
post-process the results into whatever presentation format is needed,
including perhaps re-calculating overall results (example: if we have
a list of known fails correlated to JIRA numbers, then a stylesheet
could filter out these fails and report them as known bugs instead.
|
| | |
| Section needs updating to reflect the fact that while the options the
tests see remain as below, the user must now prefix all optionNames with
'qetest.' or 'conf.', etc. when passing the options on the command line or
via my.test.properties or test.properties -sc |
Most tests can either accept options from a properties file, via:
TestName -load file.properties
or simply from the command line (which overrides the properties file) like:
TestName -arg1 value1 -arg2 -arg3 value3
To see all options, call a test with an illegal argument to force it
to print out it's .usage(). You may mix setting options from a properties
file and from the command line; command line options will take precedence.
For another description of options, see
xml-xalan\test\test.properties,
which describes most of them with comments. Remember that the prefixes
'qetest.', 'conf.' etc. are used by the Ant build.xml file to manage which
properties are used for different kinds of tests, and are ripped off before
being passed to the Java test script itself. Thus qetest.loggingLevel=99 in
the test.properties file becomes just loggingLevel of 99 when passed to the test.
| Path-like options set in a properties file generally should use
forward slashes (legal in URL's), even on Windows platforms. |
Quick list of options
- -logFile resultsFileName.xml
- sends test results to an XML-based results file
- -loggingLevel nn
- determines how much information is sent to your logFile, 0=very little, 99=lots
- -ConsoleLogger.loggingLevel nn
- determines how much information is sent just to the default ConsoleLogger:
since often you won't be watching the console as the test is running, you can set this
lower than your loggingLevel to speed the tests up a little
- -inputDir path/to/tests
- path to a directory tree of input *.xml/*.xsl files, using your system's separator
- -outputDir path/to/output/area
- where all output is sent
- -goldDir path/to/gold
- path to a directory tree of reference output - this tree should be
a parallel structure to the inputDir tree
- -category dirName
- only run this single named subdir within inputDir
- -excludes 'test1.xsl;test2.xsl'
- will skip running any specifically named tests; do not use any path elements here
- -flavor xalan|trax|trax.d2d
- which kind/flavor of Processor to test; see
ProcessorWrapper.java
- -testlet TestletClassname
- For StylesheetTestletDriver, use a different class for the testing algorithim
- -load file.properties
- (read in a .properties file, that can set any/all of the other opts)
This option is automatically used by the Ant build.xml file, so you normally
would not specify it yourself.
Note that most options work equivalently with either Xalan-J or Xalan-C tests.
When running tests using Ant, the <xalantest> task actually
marshalls various Ant variables and uses the precompiled org.apache.qetest.xsl.XSLTestAntTask
to actually execute the test (either in the same JVM or forked depending on
${fork-tests}. It simply strips the appropriate set of prefixes
off of required Ant variables and dumps them into an XSLTestAntTask.properties
file on disk, which it tells the test executing to -load.
|
|
|
