HOME > Community > Projects > xtest

XTest for NetBeans Modules

Author: Jiri Skrivanek
Last update: June 27, 2006

This document describes how to develop, customize and run tests of NetBeans modules in NetBeans IDE. It applies both for standalone modules and modules inside NetBeans CVS repository. All features and examples discussed in this document can be evaluated on sample modules. You just need to unzip this mymodule.zip into NetBeans repository and open project in IDE. For standalone module dowload this helloworld.zip. To quickly start using XTest take a look at this short D E M O.
If you want to use XTest for a Java project outside NetBeans repository, please, take a look at this document.

Note: This document is not applicable for versions lower than NetBeans 4.0.

Structure of Tests

Every test type in sense of XTest has its own packages node. It is common to have at least unit tests and functional tests. Unit tests check correctness of written code at method base. They are usually run against binaries without a need to start IDE. On the other hand functional tests check functionality of resulting application as a whole. They are usually UI tests which run on started IDE. The structure in the Projects view and the Files view is the following:

Projects view

My Module
    - Source Packages
       - org.netbeans.modules.mymodule
           - MyModuleClass.java
    - Unit Test Packages
       - org.netbeans.modules.mymodule
           - MyModuleClassTest.java
    - Functional Test Packages
       - validation
           - MyModuleValidationTest.java

Files view

My Module
    - src
       - org
          - netbeans
             - modules
                - mymodule
                   - MyModuleClass.java
    - test
       - unit
          - src
             - org
                - netbeans
                   - modules
                      - mymodule
                         - MyModuleClassTest.java
       - qa-functional
          - src
                - validation
                   MyModuleValidationTest.java

Module Test Scripts

Building and execution of tests is controlled by ant scripts. For NetBeans modules majority of stuff is hidden in templates. Then our scripts just contain import clause. If you are not satisfied with default settings imported from templates, of course you can change your build-${xtest.testtype}.xml according to your needs. But remember that classpath can be changed by extra properties as described here. Anyway if you want, you can override default compiler or executor and add definition of other XTest properties. These build scripts can be found in the Files view under the root node of the project:

My Module
    + nbproject
    + src
    - test
       - build.xml
       - build-qa-functional.xml
       - build-unit.xml

build.xml imports xtest.xml template and defines default values of XTest mandatory properties. The easiest example looks like this:

<project name="mymodule/test" basedir="." default="all">
    <!-- Name of tested module -->
    <property name="xtest.module" value="mymodule"/>
 
    <!-- Imports buildtests, cleantests, runtest, cleanresults, realclean, printconfig targets. -->
    <import file="../../nbbuild/templates/xtest.xml"/>

    <!-- default testtypes, attributes used when no value is supplied from command line -->
    <property name="xtest.testtype" value="unit"/>
    <property name="xtest.attribs" value="stable,ide,validation"/>
</project>

For standalone modules it is almost the same but we need to define xtest.home and other extra properties:

<project name="org.netbeans.helloworld/test" basedir="." default="all">
    <!-- Name of tested module -->
    <property name="xtest.module" value="mymodule"/>

    <!-- Home of XTest -->
    <property name="xtest.home" location="mywork/netbeans/xtest-distribution"/>

    <!-- Imports buildtests, cleantests, runtest, cleanresults, realclean, printconfig targets. -->
    <import file="${xtest.home}/lib/templates/xtest.xml"/>

    <!-- default testtypes, attributes used when no value is supplied from command line -->
    <property name="xtest.testtype" value="unit"/>
    <property name="xtest.attribs" value="stable,ide,validation"/>

    <!-- Points to directory with IDE to test. Optional for IDE tests. -->
    <property name="netbeans.dest.dir" location="mywork/netbeans"/>
    <!-- Points to directory with IDE to test. Optional for IDE tests. -->
    <property name="netbeans.user" location="mywork/netbeans-userdir"/>
</project>

Template xtest.xml defines buildtests, cleantests, runtest, cleanresults, realclean, printconfig targets. These targets should not be changed.

build-qa-functional.xml imports xtest-qa-functional.xml template and that's it:

<project name="mymodule/test-qa-functional" basedir="." default="all">
    <!-- Imports default qa-functional-compiler and runidetest executor.
        jemmy and jellytools jars are on classpath for both. -->
    <import file="../../nbbuild/templates/xtest-qa-functional.xml"/>
</project>

For stadalone module is the difference only in location of template:

<project name="org.netbeans.helloworld/test-qa-functional" basedir="." default="all">
    <!-- Imports default qa-functional-compiler and runidetest executor.
        jemmy and jellytools jars are on classpath for both. -->
    <import file="${xtest.home}/lib/templates/xtest-qa-functional.xml"/>
</project>

Template xtest-qa-functional.xml adds jemmy and jellytools jars to classpath and uses it both in qa-functional-compiler and runidetest targets. Additionally it loads project.properties and adds path in test.qa-functional.cp.extra property to compilation classpath and path in test.qa-functional.run.cp.extra to execution classpath.

build-unit.xml also only imports xtest-unit.xml:

<project name="mymodule/test-unit" basedir="." default="all">
    <import file="../../nbbuild/templates/xtest-unit.xml"/>
</project>

For standalone module it is again similar:

<project name="org.netbeans.helloworld/test-unit" basedir="." default="all">
    <import file="${xtest.home}/lib/templates/xtest-unit.xml"/>
</project>

Template xtest-unit.xml constructs classpath that all libraries on which mymodule depends are added and also module jar is added. The classpath is used both in default-compiler and run-unit-test targets.

Classpath Customization

If user wants to customize classpath for test compilation and execution, he can define values of extra properties which are added to classpath in templates. These properties are loaded from project.properties file which is under Important Files or in the Files view:

My Module
    - nbproject
       - project.properties
    - src

For every test type there exists an extra property for test compilation and test execution. An example of project.properties file:

# added to classpath for compilation of unit tests
test.unit.cp.extra=
# added to classpath for running of unit tests
test.unit.run.cp.extra=
# added to classpath for compilation of qa-functional tests
test.qa-functional.cp.extra=${openide/windows.dir}/modules/org-openide-windows.jar:\
${openide/util.dir}/lib/org-openide-util.jar
# added to classpath for running of qa-functional tests
test.qa-functional.run.cp.extra=

Remember that tests run in IDE (mostly functional tests) don't need module jars on classpath because they run "inside" IDE. That's why property test.qa-functional.run.cp.extra will be empty in common cases.

Whenever you are going to modify classpath, also remember that you have to use paths corresponding to cluster build schema. That means instead of fixed paths use properties like ${openide/windows.dir} to locate required jar file. Dir properties are defined in ${netbeans.dest.dir}/moduleCluster.properties.

Data for Tests

Data for tests which can include sample classes, files, projects or whatever should be stored in the directory mymodule/test/${xtest.testtype}/data. Sub folders of data can be arbitrary except golden files. Golden files has to match the following pattern data/goldenfiles/<package_hierarchy>/${classname}/${testmethodname}.pass. If you need more details about golden files, please look at this document. XTest data folder can be reached in ant scripts by ${xtest.data} property. In test cases it is available through getDataDir() method of NbTestCase class.

Build & Run Tests in IDE

Actions to compile and run tests in IDE exist in context menu on project node and in main menu hierarchy.

Context menu items on project node when any test is available:

XTest

Clean - clean tests and results. In fact it calls targets realclean from test/build.xml.

Build ${xtest.testtype} Tests - build tests using XTest build scripts. In fact it calls target buildtests from test/build.xml with parameter -Dxtest.testtype.
Run ${xtest.testtype} Tests - run tests using XTest build scripts and show results in default browser. In fact it calls target runtests from test/build.xml with parameter -Dxtest.testtype.
Measure ${xtest.testtype} Test Coverage - download Emma coverage tool (if needed), instrument test classes, run tests using XTest build scripts and show coverage report in default browser. In fact it calls target coverage from test/build.xml with parameter -Dxtest.testtype.

${xtest.testtype} is replaced by test type name (e.g. unit, qa-functional - see Structure of Tests). It means every test type has a pair of build and run actions. Property xtest.home is set to ${netbeans.user}/xtest-distribution

Main menu hierarchy items:

Test Project (Alt+F6) - run all unit tests for the project

Run File

Run File (Shift+F6) - run selected class

class under Unit Test Packages - it runs selected test class
class under "Functional Test Packages" - it is run in the same JVM as IDE to be able to exercise UI.

Test File - run appropriate unit test for selected class (e.g. run MyModuleClassTest for MyModuleClass selected)

Build & Run Tests from Command Line

Tests can be run from command line. Property netbeans.dest.dir points to NetBeans installation directory which is by default nbbuild/netbeans. Command then could look like this:

cd mymodule/test
ant -Dnetbeans.dest.dir=D:/netbeans -Dxtest.testtype=qa-functional -Dxtest.attribs=validation

If you omit all parameters, there will be used default ones as defined in test/build.xml. Target runtests is called by default. You can run additional targets like buildtests, cleantests, cleanresults, realclean.