<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- $Id: testing.xml 1298724 2012-03-09 07:55:31Z cbowditch $ -->
<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.3//EN" "http://forrest.apache.org/dtd/document-v13.dtd">
<document>
  <header>
    <title>Apache™ FOP Development: Testing</title>
    <version>$Revision: 1298724 $</version>
  </header>

    <body>
    <section id="build">
      <title>"Build" Testing</title>
      <p>Apache™ projects use an automated build tool called "gump" to create nightly builds from the SVN repository. Gump sends "nag" messages if the build fails. This can be considered a sort of basic test of the build process. To view the most recent logs of the gump builds:</p>
      <ul>
        <li><link href="http://gump.cocoondev.org/xml-fop.html">Gump build for the Trunk</link></li>
        <li><link href="http://gump.cocoondev.org/xml-fop-maintenance.html">Gump build for the Maintenance Branch</link></li>
      </ul>
    </section>
    <section id="basic">
      <title>Basic/API Testing</title>
      <p>There is a group of basic API tests that are included in the build process.
For these tests to occur, JUnit must be available to Ant (simply copy junit.jar into Ant's lib directory).
The build will then report error(s) if the high-level APIs for Driver and the Transcoders fail.
The tests do not check the output, but only ensure that something is generated and without exceptions.</p>
    </section>
    <section id="layoutengine">
      <title>Layout Engine Testing</title>
      <p>
        The <link href="http://svn.apache.org/viewcvs.cgi/xmlgraphics/fop/trunk/test/layoutengine/"><code>test/layoutengine</code></link>
        directory in the repository contains a test suite for checking the functionality of Apache� FOP's
        layout engine. For information on how to create test cases for the layout engine, please
        visit the <link href="http://wiki.apache.org/xmlgraphics-fop/HowToCreateLayoutEngineTests">Wiki page</link>.
      </p>
    </section>
<section id="functional">
  <title>Functional Testing</title>
  <warning>The "functional" testing section on this page is currently inoperative.</warning>
  <section>
    <title>Running and Using Tests</title>
    <p>
Testing is an important part of getting FOP to operate correctly and conform to the
necessary standards.
    </p>
    <p>
A testing system has been set up that works with as a build target when developing
with FOP. A developer can run the tests after making changes to the code, the aim
is to have the tests run to verfiy that nothing working has been broken.
    </p>
    <p>
To setup the testing the developer must place a reference fop.jar in the
"&lt;cvs_repository&gt;/test/reference/" directory. This jar will be dynamically
loaded to create the reference output.
    </p>
  </section>

  <section>
    <title>W3C TestSuite</title>
    <p>
The testing is set up so that you can download the testsuite from
<jump href="http://www.w3.org/Style/XSL/TestSuite/">http://www.w3.org/Style/XSL/TestSuite/</jump>,
unzip the file into the base directory of FOP.
Then you can uncomment the lines in the build.xml file in the test target and itwill run through all the tests in the testsuite distribution.
    </p>
  </section>

  <section>
    <title>Writing a Test</title>
    <p>
A test belongs to one of a few catagories. A basic test should excercise one
element in a number of situations such as changing a property. This should have
at least one normal value, one border value and one invalid value. If the property
can be of different types then this should also be included.
    </p>
    <p>
A bug test is a test that is specifically aimed at a problem with FOP. That is, the test
is not excercising the specification but rather a problem with FOP in handling a particular
situation that is not exposed with the other testing.
    </p>
    <p>
A system test is one that tests the abitlity of FOP to handle a number of different
elements together.
    </p>
    <p>
A test can consist of a complete fo document or a part of the document such as
some elements that will be placed into the flow of a standard document.
    </p>

  </section>
  <section>
    <title>Submitting a Test</title>
    <p>
If you have a test which you think would be useful you should supply the
test and a diff to the appropriate test suite xml file. Make sure that the
test works as would be expected against the current build.
    </p>
  </section>

  <section>
    <title>How Testing Works</title>
    <p>
The tests are stored in the "&lt;svn_repository&gt;/test" directory.
    </p>
    <p>
You can run the tests by specifying the build target "test" ie: <br/>
<code>ant.sh test</code> (Unix)<br/>
<code>ant test</code> (Windows)<br/>
    </p>
    <p>
This will then compare the current code in the local src directory to a specified
release of FOP. Any differences between the current code and the output from
the reference version will be reported. If the test previously passed then the
test run will have failed.
    </p>
    <p>
The testing is done by reading a test suite xml file, which corresponds to the
standard testsuite.dtd supplied from w3c. This xml file contains a test xml
file and an xsl file (which may simply copy the file). It also contains information
such as if the test has passed and any comments.
    </p>
    <p>
For FOP the testing is done by rendering all the testing documents using the
XML renderer. The XML files are then compared to see if there are any differences.
    </p>
  </section>

  <section>
    <title>SVG Testing</title>
    <p>
The testing of SVG is not part of this testing system. SVG is tested for its rendering
accuracy by using the transcoding mechanism via 
<link href="ext:batik">Apache Batik</link>. So that the only part that needs
testing is how the SVG image is embedded inside the flow of the fo document.
    </p>
  </section>
</section>
</body>
</document>