XTest in NetBeans IDE 4.0
Author: Jiri
Skrivanek
Last update: October 27, 2004
This document describes how to develop, customize and run tests in
NetBeans IDE 4.0. In this version of IDE there were made major
changes in build system and projects infrastructure. It requires new
working habits and approaches. Changes are shown on examples. If
you unzip this
mymodule.zip into NetBeans repository, you can execute tests.
Content
Tests in IDE
Every test type in sense of XTest has its own packages node. The
structure in the Projects view and the Files view is the
following:
Projects view
My Module
-
Source Packages
- org.netbeans.modules.mymodule
- MyModuleClass.java
- Unit Test Packages
-
org.netbeans.modules.mymodule
-
MyModuleClassTest.java
- Functional Test Packages
- my.functional
-
MyFunctionalTest.java
Files view
My Module
- src
- org
- netbeans
- modules
- mymodule
- MyModuleClass.java
- test
- unit
- src
- org
- netbeans
- modules
- mymodule
- MyModuleClassTest.java
- qa-functional
- src
- my
- functional
MyFunctionalTest.java
Actions
Actions to compile and run tests in IDE exist in main menu hierarchy
and
context menu on project node.
Main menu hierarchy items:
- Run
- Run Other
- Test
Project - run
all unit tests for the project
- Run
File - run selected class
- class under Unit Test Packages - it runs selected
test class
- class under "Functional
Test Packages" - it is run in the same JVM as IDE. It enables to access
IDE API
inside tests.
- Test
File - run appropriate unit test for selected class (e.g. run
MyModuleClassTest for MyModuleClass selected)
Context
menu items on project node:
- Run Unit
Tests - run all unit tests for the project Alt+F6 (the same as
Test project in main menu)
- Run Unit
Tests using XTest - run unit tests using XTest build scripts and
show results in default browser. In fact it calls targets cleanresults, runtests and show-results-nb from test/build.xml with parameter -Dxtest.testtype=unit.
- Build
Functional Tests -
build functional tests using XTest build scripts. In fact it calls
target buildtests from test/build.xml with parameter -Dxtest.testtype=qa-functional.
- Run
Functional Tests using XTest - run functional tests using XTest
build scripts and show results in default browser. In fact it calls
targets cleanresults, runtests and show-results-nb from test/build.xml with parameter -Dxtest.testtype=qa-functional.
Customize classpath
If user wants to customize classpath for test compilation and
execution,
he can define values of extra properties which are added to classpath
in templates. These properties are load from
project.properties file in the
Files view:
My Module
- nbproject
- project.properties
- src
For every test type there exist an extra property for test
compilation and test execution:
- test.unit.cp.extra -
added to classpath for compilation of unit tests
- test.unit.run.cp.extra
- added to classpath for running of unit tests
- test.qa-functional.cp.extra -
added to classpath for compilation of qa-functional tests
- test.qa-functional.run.cp.extra
- added to classpath for running of qa-functional tests
An example of
project.properties
file:
test.unit.cp.extra=
test.unit.run.cp.extra=${openide/execution.dir}/${nb.modules.dir}/org-openide-execution.jar:${openide.dir}/${nb.lib.dir}/openide.jar
test.qa-functional.cp.extra=${openide.dir}/${nb.lib.dir}/openide.jar
test.qa-functional.run.cp.extra=
Remember that tests run in IDE (mostly functional tests) don't need
module jars on classpath because they run "inside" IDE. That's why
property
test.qa-functional.run.cp.extra
will be empty in common cases.
Whenever you are going to modify classpath, also remember that you have
to
use paths corresponding to cluster build schema. That means instead of
fixed paths use properties like
${openide/execution.dir},
${nb.modules.dir}... to
locate required jar file. Also be aware that module jar files were
renamed to follow package hierarchy, e.g.
openide-execution.jar was
renamed to
org-openide-execution.jar.
Dir properties are defined in
nbbuild/directories.properties
and
nbbuild/netbeans/moduleCluster.properties.
Module test scripts
Structure of scripts used for test execution remains the same. But
content becomes much more simpler because of templates usage. These
build scripts can be found in the Files view under the root node of the
project:
My Module
+ nbproject
+ src
- test
- build.xml
- build-qa-functional.xml
- build-unit.xml
build.xml
imports xtest.xml template and defines default values of XTest
mandatory properties. The easiest example looks like this:
<project name="mymodule/test"
basedir="." default="all" >
<!-- Name of
tested module -->
<property
name="xtest.module" value="mymodule"/>
<!--
Imports buildtests, cleantests, runtest, cleanresults, realclean,
printconfig targets. -->
<import
file="../../nbbuild/templates/xtest.xml"/>
<!--
default testtypes, attributes used when no value is supplied from
command line -->
<property
name="xtest.testtype" value="qa-functional"/>
<property
name="xtest.attribs" value="stable"/>
</project>
Template
xtest.xml defines
buildtests,
cleantests,
runtest,
cleanresults,
realclean,
printconfig
targets. These targets should not be changed.
build-qa-functional.xml
imports
xtest-qa-functional.xml
template and that's it:
<project
name="mymodule/test-qa-functional" basedir="." default="all">
<!--
Imports default qa-functional-compiler and runidetest executor.
jemmy and jellytools jars are on classpath for both.
-->
<import
file="../../nbbuild/templates/xtest-qa-functional.xml"/>
</project>
Template
xtest-qa-functional.xml
adds
jemmy and jellytools jars to classpath and uses it both in
qa-functional-compiler
and
runidetest targets.
Additionally it loads project.properties and adds path in
test.${xtest.testtype}.cp.extra
property to compilation classpath and path in
test.${xtest.testtype}.run.cp.extra
to execution classpath
.
build-unit.xml
also only
imports
xtest-unit.xml:
<project
name="mymodule/test-unit" basedir="." default="all">
<import
file="../../nbbuild/templates/xtest-unit.xml"/>
</project>
Template xtest-unit.xml
constructs
classpath that all libraries on which mymodule depends are added and
also module jar is added. The classpath is used both in
default-compiler and
run-unit-test targets.
If you are not satisfied with default settings imported from templates,
of course you can change your build-xxx.xml according to your needs.
But remember that classpath can be changed by extra properties as
described
here. Anyway if you
want, you can override default compiler or executor and add definition
of other XTest properties.
Command line test
execution
Test can be run from command
line the same way as before. The only change is that to point to IDE,
you have to use
netbeans.dest.dir
property instead of
netbeans.home.
The property
netbeans.dest.dir
is the root of NetBeans clustered build. Command then could look like
this:
cd mymodule/test
ant -Dnetbeans.dest.dir=D:/builds/041027/netbeans
-Dxtest.testtype=qa-functional
-Dxtest.attribs=validation
XTest compile output
directory
XTest was used to compile tests next to sources in src directory. But
it is suitable to compile tests to a separate directory. Tests are be
compiled to
mymodule/build/test/${testtype}/classes.
The structure as can be seen in the Files view looks like this:
My Module
- build
- classes
- test
- qa-functional
- classes
- unit
- classes
+ nbproject
+ src
+ test
This change doesn't need any user action.
XTest data
In order to separate sample data from sources, it was introduced a new
schema for storing such a data. Test data which can include sample
classes, files, projects or whatever should be stored in the
directory
mymodule/test/${xtest.testtype}/data.
Sub folders of data can be arbitrary except golden files. Golden files
has to match the following pattern
data/goldenfiles/${classname}/${testmethodname}.pass.
If you need more details about golden files, please look at this
document.
XTest data folder can be reached in scripts by
${xtest.data} property. In test
cases it is available through
getDataDir()
method in
NbTestCase
class. Old approach when data are stored within test source is obsolete
but it still works and you don't need to rewrite everything if you have
a reason to keep it.
