XTest Driver Description

Author: Libor.Martinek (libor.martinek@sun.com )
Date: February 12, 2003

---

What is this document for?

This document is intended for someone who wants execute all existing tests automatically (e.g. daily). This document is not intended for test developers.

Overview

Driver is tool which helps automate whole process of xtest execution. Driver is responsible for installing IDE, checking out tests, building and runing them. Tests are executed by xtest instances. Driver can execute more xtest instances.

Driver is configured by properties. The best way is to use property file. Property file (also known as configuration file) should contain properties mentioned below and also it can contain other properties used in xtest itself.

Driver organization

Driver files are stored in CVS under xtest module: main buildfile is xtest/lib/driver.xml and task sources are under xtest/src/org/netbeans. Because driver configuration is site-specific, you have to run driver from wrapper which sets some properties and call main driver's buildfile. Wrapper can be named as you want, best would be build.xml. Driver wrapper looks like this:

<?xml version="1.0"?>
<project name="Xtest driver" basedir="." default="start">

   <property name="xtest.home" location="xtest-dist"/>
   <property name="ide.installto" location="ide" />
   <property name="xtest.driver.config" location="driver.properties"/>
   <property name="xtest.results" location="results"/>
   <property name="xtest.workdir" location="work"/>
   <property name="xtest.driver.backup" location="backup"/>


   <target name="start">
        <ant dir="." antfile="${xtest.home}/lib/driver.xml" target="start"/> 
   </target>
</project>

This wrapper has to have three propertis:

• xtest.home - which points to place xtest distribution is installed
• ide.installto - which specifies directory where IDE will be installed
• xtest.driver.config - which points to driver propertyfile

Running driver

1. Create directory for driver wrapper.
2. Copy (or write) driver wrapper to newly created directory.
3. Edit the three properties in wrapper.
4. Copy (or write) driver property file, e.g driver.properties.
5. Edit driver property file (see description in below chapters).
6. Run driver

To run driver just type ant in driver wrapper directory. You can override properties defined in driver wrapper or driver property file by adding properties into commandline: ant -Dxtest.driver.config=my.properties -Dide.installto=myide -Dide.install.path=/builds/last_NB.zip

Driver property file

See example of driver property file.

Setting execute actions

You can choose what driver will do by property xtest.execute . This property contains list of these values:

• install-ide - will install IDE and set property netbeans.home . Remember when you skip "install-ide" then property netbeans.home will not be set and building can fail if it requires this property.
• checkout - will checkout system modules and test modules
• execute - will build all test in all xtest instances

Example:

xtest.execute=install-ide,checkout,execute

- do everything

xtest.execute=install-ide,checkout

- install IDE and only checkout tests. Don't use checkout independently, because some tests requires property netbeans.home which is set in install-ide.

xxtest.execute=install-ide,execute

- only install IDE and execute all tests. It supposes that tests were already checked out.

Installing IDE

IDE can be installed by three ways:

• from .zip
• from .zip downloaded via http
  
• from .exe or .bin installer
• from .jar installer
• use already installed IDE

What installation type will be used says property ide.install.type , which can have values "zip", "http-zip", "installer", "jarinstaller" or "dir". Property ide.install.path points to file (or directory) used for installation.

If you use "installer" or "jarinstaller" type, you can also set additional properties: installer.java.home, which point to java to use for installer, property installer.params , which can contain other parameters and property installer.args , which can contain arguments put to installer command line. Consult installer manual for more details.

Examples:

ide.install.type=zip
ide.install.path=c:\\builds\\last_NB.zip

or

ide.install.type=http-zip
ide.install.path=http://www.netbeans.org/download/dev/night/build200207090100/NetBeansIDE-dev-200207090100.zip

or

ide.install.type=installer
ide.install.path=c:\\install\\ffj_ce_win32_en.exe
installer.java.home=c:\\java\\jdk1.3.1
installer.params="pointbaseInstall=yes"
installer.args="-is:log install.log"

or

ide.install.type=jarinstaller
ide.install.path=c:\\install\\ffj_ce_en.jar
installer.java.home=c:\\java\\jdk1.3.1
installer.params="-DpointbaseInstall=yes"

or

ide.install.type=dir
ide.install.path=c:\\netbeans

Where to install

By default IDE is installed to directory "ide". You can say where to install IDE by property ide.installto.

Installing NBMs

It's possible to add NBMs to IDE. It's not real installation of NBM, becasue it only unzip NBM to IDE installation directory and not execute any installation script. Currently it's only possible to install NBM which are zipped in one zip.

Set these properties:

• ide.nbm.zipfile - path to zip with NMBs.
• ide.nbm.list - list of NBMs to install

Example:

ide.nbm.zipfile=/builds/nbms/last_NBMs.zip
ide.nbm.list=nb/mdrapi.nbm,nb/mdr.nbm,nb/jmi.nbm,nb/jmitoolkit.nbm,nb/mof.nbm

Setting xtest instances

Xtest driver can execute tests from more xtest instances. Each instance has to be defined in driver's property file.

Examples:

xtest.instance.cvs.workdir/1=c:\\testing\\nb_all
xtest.instance.location/1=xtest/instance
xtest.instance.cvs.root/1=cvs.netbeans.org:/cvs{trunk}
xtest.instance.config/1=config-1

xtest.instance.cvs.workdir/2=c:\\testing\\xy_all
xtest.instance.location/2=myrepository/tests/xtest-instance
xtest.instance.testroot/2=myrepository/tests
xtest.instance.cvs.root/2=othercvs:/cvs{trunk}
xtest.instance.config/2=config-name
xtest.instance.master-config/2=c:\\my-master-config.xml

Each instance has to have four mandatory properties and can have one optional. Name of these properties have to consist of given prefix and arbitrary postfix which determines particular instance. In example above first instance has postfix "/1" and second instance has postfix "/2".

Mandatory properties (prefixes):

• xtest.instance.cvs.workdir - Workdir for CVS check out. Enter only root of the workdir even if tests are stored under some subdirectories. Use xtest.instance.testroot property for specifying this sudirectory
• xtest.instance.location - Path to instance. Use relative path to workdir and use only slashes ("/"). Xtest instance is checked out from CVS using this path.
• xtest.instance.cvs.root - Default CVS root and branch. Use this form: <cvs_repository>{<cvs_branch>}
• xtest.instance.config - Name of config to run

Optional property (prefix):

• xtest.instance.testroot - Root of tests relative to workdir. Again use slashes ("/") because this root is added before tested module as prefix and used for checkout.
• xtest.instance.master-config - Master config which should be used instead of default master-config.xml in instance directory. If this points to file outside instance directory, then this file isn't checked out from CVS and must be present locally.
• xtest.instance.re.conf.prefix - It's possible to use file re.cof which describes which module is checked out with which tag. You need to set property xtest.driver.re.conf (and optionally xtest.driver.re.user.conf) with path to re.conf (user.conf) and set instance property xtest.instance.re.conf.prefix to prefix of properties in re.conf which will be used.

Checking out

Every repository has to have assigned local workdir. This is done by property xtest.cvs.repositories. It contains list of pairs cvs_repository and local_directory which are separated by "-". It has this form :

xtest.cvs.repositories=<cvs_repository1>-<local_directory1>,<cvs_repository2>-<local_directory2>,...

Example:

xtest.cvs.repositories=cvs.netbeans.org:/cvs-c:\\testing\\nb_all,othercvs:/cvs-c:\\testing\\xy_all

Every module which will be checked out can have defined branch and repository distinct default ones. Definition is done by property in form "<module>.branch=<cvs_repository>{<cvs_branch>},..." . Identifier <module> is name of module including package which will be checked out, for example "openide/test". Definition of one module can contain more repositories separated by comma.

Example:

openide/test.branch=cvs.netbeans.org:/cvs{trunk}
somemodule.branch=cvs.netbeans.org:/cvs{trunk},othercvs:/cvs{trunk}

Checkout consists from two phases: checkout system modules and checkout tests.

System modules are modules like xtest, junit etc. List of these modules are taken from property xtest.driver.sysmodules . These modules can also have defined branches.

Example:

xtest.driver.sysmodules=nbbuild,junit

nbbuild.branch=cvs.netbeans.org:/cvs{trunk}
junit.branch=cvs.netbeans.org:/cvs{trunk}

If property xtest.driver.sysmodules isn't defined then checkout of system modules isn't performed.

Setting information necessary for further processing by publishing server

If you plan to integrate results from severals builds of product, from several tested products or from several testing machines, you need to use publishing server of XTest. In this case you need to supply the following properties, so the results can be correctly processed and integrated together by the publishing server.

• xtest.tested.project - which project/product is being tested - usually project and CVS branch (e.g. NetBeans release33, NetBeans dev JDK 1.3, ...).
• xtest.tested.project_id - project ID used when storing results to database.
• xtest.testing.group - which group runs the tests, for example QA, RE, Development, ...
• xtest.testing.team - which team runs the tests. This property is optional. If not entered, default team defined in publishing server will be used.
• xtest.tested.type - which kind of tests are primarily run. For example QA will usually run functional tests, development unit tests, etc.
• xtest.ship.results.to - path where publishing server expects results from testing machines. Usually this is some shared directory (via NFS for Unix machines, via SMB for Windows machines), where all testing machines save zip archive with results at the end of testing.
• xtest.machine - full name of the testing machine. This property needs to be specified when results collected from several domans are collected together, because XTest is able to get only hostname without domain name for the particular testing machines. If you do not plan to integrate results from several domains, you don't have to set this property.