HOME > Community > Projects > xtest

How to write GUI tests for XTest

Author: Jiri Skrivanek
Last Update: August 26, 20023

This document should explain how to write GUI tests for JUnit/XTest infrastructure. GUI tests use Jemmy and Jellytools libraries. Simple examples can be found in xtest/examples/MyModule/test/gui directory.

Directory structure

Tests have to be stored in defined structure which follows the pattern:

${module}/test/{testtype}/src/*${package}*/${testclass}

Example of such structure could be:

${module}/test/gui/src/guitest/
GuiTest.java
GoldenGuiTest.java
${module}/test/gui/src/guitest/data/goldenfiles/GoldenGuiTest/testPart1.pass

Tests written by QA people should have qa as prefix, for example qa-functional and so on. Only java files have to be present in CVS repository because XTest infrastructure satisfies that files are compiled before their execution. Golden files are stored in a separate directory dedicated for resources. It is called data and it is in the same package as a testclass. Golden files are then under goldenfiles package in data. You can learn more about golden files in golden files guide.

Test implementation

Here is an example of simple GUI test class:

import org.netbeans.junit.NbTestSuite;

import org.netbeans.jellytools.JellyTestCase;
import org.netbeans.jellytools.NbDialogOperator;
import org.netbeans.jellytools.RuntimeTabOperator;
import org.netbeans.jellytools.actions.Action;
import org.netbeans.jellytools.actions.PropertiesAction;
import org.netbeans.jellytools.nodes.Node;
import org.netbeans.jellytools.properties.Property;
import org.netbeans.jellytools.properties.PropertySheetOperator;

public class GuiTest extends JellyTestCase {
    
    public GuiTest(String testName) {
        super(testName);
    }
    
    public static void main(java.lang.String[] args) {
        junit.textui.TestRunner.run(new NbTestSuite(GuiTest.class));
    }
    
    public void testPart1() {
        Node rootNode = RuntimeTabOperator.invoke().getRootNode();
        Node debuggerNode = new Node(rootNode, "Debugger");
        new PropertiesAction().perform(debuggerNode);
        PropertySheetOperator pso = new PropertySheetOperator("Debugger");
        Property p = new Property(pso, "Debugger State");
        String state = p.getValue();
        pso.close();
        String expectedState = "not running";
        assertEquals("Debugger in wrong state", expectedState, state);
    }
    
    public void testPart2() {
        new Action("Help|About", null).perform();
        new NbDialogOperator("About").close();
    }
}

In fact, these are two test cases. In JUnit a test case is represented by a test method within a test class. Test class has to extend org.netbeans.jellytools.JellyTestCase class (read more about it here) and has to also include constructor with String parameter. JUnit convention forces to start name of executing method with "test" prefix. Valid method names are testThisFeature(), testThatFeature() and so on.
First test case (testPart1() method) opens the Runtime explorer and opens Properties Window of Debugger node. Then it checks status of debugger.
Test case uses self-checking approach. Jemmy output is used only for back-tracking in case of failure. By default it is redirected by JellyTestCase to file jemmy.log in working directory. You can use assertXXX() or fail() methods to indicate a failure (not expected state) within the test. In addition if something goes wrong (timeout expires or something similar), runtime exception is thrown from jemmy and caught in JellyTestCase. Depending on system properties, JellyTestCase creates screen shot in working directory and closes all modal dialogs.

Test method using golden files would differ only in checking of expected results:

    public void testPart1() {
        Node rootNode = RuntimeTabOperator.invoke().getRootNode();
        Node debuggerNode = new Node(rootNode, "Debugger");
        new PropertiesAction().perform(debuggerNode);
        PropertySheetOperator pso = new PropertySheetOperator("Debugger");
        Property p = new Property(pso, "Debugger State");
        String state = p.getValue();
        pso.close();
        // write to ref PrintStream to compare with golden file
        ref(state);
        // compare outpup written to ref PrintStream with contents of golden file
        // (data/goldenfiles/GoldenGuiTest/testPart1.pass)
        compareReferenceFiles();
    }

In the body you can use ref(String) method to collect reference output. At the end you have to call compareReferenceFiles() method. It will compare output collected by ref() methods with contents of golden file data/goldenfiles/GoldenGuiTest/testPart1.pass (pattern is data/goldenfiles/${classname}/${methodname}.pass). If files differ, test is failed, otherwise it is passed. You can also use assertXXX() or fail() methods to indicate a failure within the test. You can learn more about golden files in golden files guide.

Running GUI tests

Inside IDE

It is obvious you are developing and debugging your tests inside IDE. In order to run GUI test you need to have several filesystems mounted:

${module}/test/{testtype}/src	sources of tests (e.g. ${module}/test/gui/src)
jemmy.jar	latest Jemmy library
jelly2-nb.jar	latest Jelly2 library
junit.jar	distribution of JUnit
nbjunit.jar	NB extension to JUnit
other	other jars necessary to execute developed tests (like openide.jar and other)

GUI tests have to be run in the same JVM as the IDE, that is by internal executor. You can change the executor either for one class or set internal executor as default within IDE. In the first case open Properties on a class and in the Execution tab select Internal Execution in combo box of Executor property. In the second case open Tools->Options from the main menu, go to Editing->Java Sources and again select Internal Execution in Default Executor property.

If the test writer would like to use logging or golden files (see guide), s/he will have to define the nbjunit.workdir. It has to be defined in ${user.home}/junit.properties file, in the nbjunit.workdir property (e.g. nbjunit.workdir=/tmp/mytestworkdir). See NbJUnit recognized properties for details. For the XTest harness users, this property will be supplied automatically by the XTest framework.

By XTest infrastructure

Execution of tests is driven by configuration files in ${module}/test directory. Only difference against API tests is that a path to jemmy.jar and jelly2-nb.jar has to be added to build-gui.xml.

Configuration file determining which tests will be executed will have also similar structure. For GUI tests there will always be executor set to "ide". Here is an example of such config file (cfg-gui.xml):

<mconfig name="GUI tests examples config">

    
    <testbag testattribs="all,stable" executor="ide" name="All GUI tests">
        <testset dir="gui/src">
            <patternset>
                <include name="**/*"/>
            </patternset>
        </testset>
    </testbag>

    <resultsprocessor name="gui" antfile="build.xml" target="test_report" default="true"/>
    <executor name="ide" antfile="build-gui.xml" target="runidetest"/>
    <compiler name="gui-compiler" antfile="build-gui.xml" target="gui-compiler" default="true"/>

</mconfig>

You can find all above examples in xtest/examples/MyModule/test/ directory. To run tests by XTest framework, you need to check out this directory from CVS. You also need to have jemmy.jar and jelly2-nb.jar available in the following structure:

nb_all/jellytools/builds/jelly2-nb.jar
nb_all/jellytools/builds/jemmy.jar
nb_all/xtest/*

Now go to nb_all/xtest/examples/MyModule/test and run:

ant -Dxtest.testtype=gui -Dnetbeans.home=<your_netbeans_home>

You should see a lot of output and IDE should be started after a while. After tests are finished you can check results at location printed out at the end of console output. Usually it is: nb_all/xtest/examples/MyModule/test/results/index.html.

xtest

Project Home
Memebership
Announcements
Discussion Forums
Mailing Lists
Documentation & Files
Version Control
Issue Tracker

NetBeans.org