NbJUnit - unique workdir, logging and golden files guide

---

Author: Jiri Skrivanek
Last update: April 30, 2006

---

Unique workdir

For some reasons it is useful to access a unique working directory in NbJUnit test. Tests inheriting from org.netbeans.junit.NbTestCase are able to use the following methods:

    File getWorkDir() throws IOException
    public void clearWorkDir() throws IOException;

Method getWorkDir() returns a file object, which points to the test method's unique working directory. If the directory cannot be created, the method throws an IOException.

The unique working directory will be created in the following pattern:

${nbjunit.workdir}/${classpackage}.${classname}/${testmethodname}

${nbjunit.workdir}...the root of all work directories (e.g. /tmp/mytestworkdir)
${classpackage}...package path (e.g. org.netbeans.test)
${classname}...name of the class without the package specification (TestClass1)
${testmethodname}...the name of the testing method (testMethod1)

Method clearWorkDir() clears the working directory (all files including subdirectories). In the case of any problems, it throws an IOException.

If the test writer would like to use this feature, he/she may wish to define the nbjunit.workdir. It has to be defined in the ${user.home}/junit.properties file, in the nbjunit.workdir property (e.g. nbjunit.workdir=/tmp/mytestworkdir). See NbJUnit recognized properties for details. For the XTest harness users, this property will be supplied automatically by the XTest framework. If not otherwise defined, it is ${java.io.tmpdir}/tests.

Here is a simple usage example:

    public void testMethod1() throws java.io.IOException {
        // get path to my unique workdir; when it is not created, create it
        File workDir = getWorkDir();
        // do something in my workdir
        (new File(workDir,"testfile.xx")).createNewFile();
        (new File(workDir,"testfile.xy")).createNewFile();
        // well, clear all the files in the workdir
        clearWorkDir();
        // do something else in the workdir
        (new File(workDir,"testfile2.xx")).createNewFile();
        (new File(workDir,"testfile2.xy")).createNewFile();
    }

Logging  facility

It is useful to have the ability to store log messages into a file. Using a unique working directory gives us a place where every test case can store such information. Users can use these methods:

    public void log(String message)
    public void log(String filename, String message)
    public PrintStream getLog()
    public PrintStream getLog(String filename)

When no filename is supplied, the log file is created in the working directory and named ${testmethodname}.log. The log file is created for the first time when one of the above mentioned methods is called within a test method. The second time, it reuses the previously created file.

A test method then could look like this:

    public void testMethod1()  {
        log("message to log file ${workdir}/testMethod1.log");
        log("mySpecialLog.log", "message to log file mySpecialLog.log");
        // here write body of the test
        log("finished successfully.");
    }

Golden files

Comparing two files to decide whether a test passed or failed is a common testing technique. One file, known as the golden file, contains expected output. During the test execution, output is collected to a second file. At the end, these two files are compared and if they differ the test is declared as failed, otherwise as passed. It is also possible to store the output from comparing (e.g. from diff command) to another file. It can help the test developer to investigate the cause of a failure. Methods supporting this facility are:

    public void ref(String message)
    public PrintStream getRef()
    public File getGoldenFile()
    public File getGoldenFile(String filename)
    public void compareReferenceFiles()
    public void compareReferenceFiles(String testFilename, String goldenFilename, String diffFilename)

The first time ref() or getRef() methods are used in a test method, a new file is created in the unique working directory and named ${workdir}/${testmethodname}.ref. At the end of each test method this file should be compared with the golden file. The golden file has to be stored under the data folder corresponding to the following pattern:

test/${xtest.testtype}/data/goldenfiles/${package-hierarchy}/${classname}/${testmethodname}.pass

Then you can use the compareReferenceFiles() method without parameters. Otherwise you need to supply the pathnames as parameters.
If you want to store the differences from comparing (if any), you need to set an external diff command because the built-in implementation doesn't produce any output. For example, add the following to the ${user.home}/junit.properties file (See NbJUnit recognized properties for details):

    nbjunit.diff.impl=org.netbeans.junit.diff.NativeDiff
    nbjunit.diff.impl.CmdLine=/usr/bin/diff -bw %TESTFILE% %PASSFILE%

The output from the diff command will be stored in the ${workdir}/${testmethodname}.diff file.

Here is an example of a test method:

    public void testMethod1() {
        ref("Output to ref file called ${workdir}/testMethod1.ref");
        // here write body of the test
        boolean somethingWrong = false;
        if(somethingWrong) fail();
        assertTrue(!somethingWrong);
        ref("another output to compare to golden one");
        compareReferenceFiles();
    }

As you see in the example, you can also use assertTrue() or fail() methods to indicate a failure. It is not recommended to put the compareReferenceFiles() method into tearDown() because the exception thrown by compareReferenceFiles() in tearDown() will appear in the result log and you will never know that there was another exception in your testXXX(), causing the files to be different.