logolineright
bottomhttp://xml.apache.org/http://www.apache.org/http://www.w3.org/
join
Overview
Getting Started
separator
Java API
separator
FAQ
Running Tests
Writing New Tests
Test Standards
Xalan-C Tests
separator
Xalan-J 2.x
Xalan-J 1.x
Xalan-C 1.x
close
In progress
 

We need to start a FAQ for testing Xalan, unfortunately the faq doctype doesn't properly work in this tree, so I'm using an s1/s2 doc instead temporarily. Some topics include:


Prepare to run tests
 

CVSROOT=:pserver:user@cvs.apache.org:/home/cvs etc.
or CVSROOT=:pserver:anoncvs@cvs.apache.org:/home/cvspublic etc.
cvs checkout xml-xalan/java
cvs checkout xml-xalan/test
cd xml-xalan/java
build jar
cd ../test
build jar
build -projecthelp - to get a list of targets


Run conformance tests
 

The conf set of tests can be run in a variety of ways, especially with each different flavor. For a list of flavors, see xml-xalan/test/java/src/org/apache/qetest/xslwrapper/TransformWrapperFactory.properties

  • build conf
    (default: trax.systemId)

  • build conf -Dconf.flavor=trax.sax
    (uses SAX IMPORTANT!)

  • build conf -Dconf.flavor=trax.dom
    (uses dom IMPORTANT!)

  • trax.file - File object instead of systemId

  • trax.stream - InputStreams - IMPORTANT!

  • trax.localPath - uses local paths instead of URLs, experimental

  • trax.systemId3 - does systemId transform three times in a row, experimental

  • process - uses command line class - IMPORTANT!


Run contrib/perf tests
 

Same as running conf tests, except substitute 'contrib' for 'conf' everywhere, or 'perf' instead. Note that 'perf' uses a custom Testlet testing algorithim that iterates several times and outputs custom perf elements.

  • build contrib -Dcontrib.flavor=trax.sax


  • build perf -Dperf.flavor=trax.stream



Run XSLTC-related tests
 

Note that currently some of the XSLTC-specific tests are built with separate targets; in the future as the XSLTC engine is more completely integrated into the Xalan core, we hope to merge the tests as well.

Note that many of the tests that use TrAX/JAXP should work equally well with XSLTC since they have their own version of a TransformerFactory; remember to set the appropriate system property or classpath appropriately. The various org.apache.qetest.trax API tests should also work normally with XSLTC.


Run XSLTMARK perf tests
 

Xalan has it's own more detailed automation harness for running stylesheet tests and capturing performance metrics. You can run the XSLTMARK set of stylesheets using Xalan's harness to see more detailed performance data.

  • Copy xsltmark/testcases to xml-xalan/test (so it's xml-xalan/test/testcases)

  • Copy xsltmark/testcases/default.conf to xml-xalan/test/xsltmark.filelist

  • cd xml-xalan/test

  • Search-and-replace xsltmark.filelist 't=' 't=testcases/'

  • Search-and-replace xsltmark.filelist 'e=' 'e=testcases/'

  • build perf -Dperf.fileList=xsltmark.filelist


  • java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl PerfScanner.xsl -out results-perf/PerfReport.html


  • (where blah includes xml-apis.jar;xalan.jar;xercesImpl.jar)


View results in HTML
 

Every test creates a results.xml (or TestName.xml) file of all the results the test has performed - no need to examine the console, since all the info will be here. We have a couple of prototype stylesheets to view the results in a semi-pretty HTML style.

There is also a tableResults.xsl stylesheet for creating a table of conformance test results. For documentation on how to use the stylesheet, please consult the section How-to: View Results.

  • build perf

    (Creates results-perf/results.xml)
  • java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl FailScanner.xsl -out results-perf/FailReport.html


  • java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl PerfScanner.xsl -out results-perf/PerfReport.html


  • build alltest

    (Creates a LOT of results in results-alltest/**)
  • build scan

    (Runs ResultScanner by default on results-alltest/**)
  • build scan -Dscan.outputDir=results-perf
    etc.
  • java -classpath blah;testxsl.jar org.apache.qetest.xsl.ResultScanner results-alltest

    (This uses ResultScanner.xsl to style all results in the whole tree under results-alltest into a single ResultReport.html in the current directory; it currently uses FailScanner.xsl to only include fail results)

Include/Exclude tests
 

Run just a subset of tests, or exclude tests using simple command line options.

  • build conf -Dconf.category=axes;boolean
    - Only run those directories

  • build conf -Dconf.excludes=axes107.xsl;boolean12.xsl
    - Skip those explicit xsl filenames


Run a single conf test
 
  • build conf.one -Dconf.test=axes44 -Dconf.flavor=trax.stream

    This will run just the axes44.xsl test, using the normal Testlet algorithim and using whatever flavor you choose (which makes it easy to see if single tests run properly using SAX, DOM, streams, whatever)

Run all API tests
 
  • build api -DtestClass=TransformerAPITest

  • This runs all the available specific API tests, which includes all the API tests in the smoketest, plus other tests that fail due to known bugzilla reports.
  • Note that in the case of
    testClass
    , you do not prefix the name of the option with conf,api,perf,etc.


Run a LOT tests
 
  • build alltest
    (runs all,alltest.other,alltest.conf,alltest.contrib)
  • build alltest.other
    runs all API tests, extensions, bugzilla, threading, and perf tests
  • build alltest.conf
    run all flavors of conf tests
  • build alltest.contrib
    run all flavors of contrib tests
  • build alltest.features
    run all conf and contrib tests, matrixed over all flavors, twice: once with the feature incremental set to true, once with optimize set to false (note: not all features matter with all flavors, but we run them all anyway. Several meg of output!)

Debug the tests
 

It's quite simple to run the tests standalone, without using Ant - this is suitable for running in your debugger or what-not (or in environments where you feel that the overhead of running Ant might interfere with the test).

Normally, we have the build.bat/.sh files and the Ant script build.xml manage parsing the command line and setting up the CLASSPATH and options for running tests. However all tests can be run on the command line, and will accept options in either a -load file.properties block, or on the command line itself.

  • debugapi.bat org.apache.qetest.trax.TransformerAPITest

    Runs just one API test without using Ant; edit debugapi.properties to change parameters passed to the test.
  • debugconf.bat [options]

    Runs a set of conf/ stylesheet tests using the normal StylesheetTestletDriver; see debugconf.properties for settable options. Note that this method does not directly support 'conf.one' to run only a single test.
  • Setup everything manually. Minimal setup is: CLASSPATH has xalan.jar;serializer.jar,xercesImpl.jar;xml-apis.jar;testxsl.jar, command line parameters or -load file.properties includes other test properties, like inputDir, goldDir, outputDir, logFile, etc.


dot
Copyright © 2000 The Apache Software Foundation. All Rights Reserved.