Signature Tests Support Framework
Design Specification

Version: 0.2
Author: Daniel Pfeffer

Abstract: This document describes design of  general "signatures compatibility tests support framework", which is a part of Xtest. This framework is designed to support NetBeans compatibility tests.

Document History:

23 Feb 2001 : version 0.2, created

2 Mar 2001 : version 0.2, updated

 

 

Related documents

XTest  Design Specification

Contents:

Introduction

Description Of The Design

Input and Output

Data

Configuration

Processing

Error Conditions

Initialization and Shutdown

Unknowns

Constraints

Introduction

Signature level compatibility tests support framework is a part of XTest. It provides capability to compare Java elements signatures in order to identify compatible and not compatible changes. The signature tests framework will be used in NetBeans Compatibility Kit primarily, but can be used generally.

Following Java elements are used to check the signatures compatibility:

To determine the compatible and not compatible changes access modifiers (default, public, protected, private) will be used.

There are two basic change types: add, remove
It is possible to define derived change type: change
which occurs when one element with the same name is removed and added (changed).
 (Note: when one element with the same name is removed and more then one added, one should be voted as a successor indicating change, and the others are identified as added. The successor identification has to be based on some heuristics ( nearest number of parameters, return type...   )

Signatures changes can be generally classified in two categories:

• Compatible changes changes which does not break backwards compatibility of the code
• Not compatible changes changes which breaks backwards compatibility of the code and can cause conflicts in dependent modules

Description Of The Design

Changes compatibility

There can be recognized two types of signatures compatibility:

• package internal
• package external 

Package internal compatible changes are compatible externally, but some externally compatible changes are not compatible within package.  

Compatibility of changes can be defined by  following tables: 

Package internal changes compatibility

elements of

access modifiers

added

removed

modified

interface

public

not compatible

not compatible

not compatible

class

public

compatible

not compatible

not compatible

class

protected

compatible

not compatible

not compatible

modifier:
-> public
compatible

class

<default>

compatible

not compatible

not compatible

-> public
compatible

class

private

compatible

compatible

compatible

Package external changes compatibility

elements of

access modifiers

added

removed

modified

interface

public

not compatible

not compatible

not compatible

class

public

compatible

not compatible

not compatible

class

protected

compatible

not compatible

not compatible

modifier:
-> public
compatible

class

<default>

compatible

compatible

compatible

class

private

compatible

compatible

compatible

Note: Abstract methods in abstract classes should be handled as interfaces. 

Implementation

The signature tests implementation is designed to fit in XTest general tests framework. The data processing is described in following diagram.

The Signature test functional unit will be based on NetBeans File systems API and Reflection or Java Hierarchy API.

Signature tests should be possible to execute from Ant. For this purposes is special signaturetest Ant task designed. signaturetest task has "tested" and "reference" sub elements defining corresponding file sets using Ant style "path-like structures". Task "signaturetest" has also "output" attribute containing test result file name. See Ant script level configuration for more details.

Input and Output

The signature tests can be seen as a black box, with the tested and reference code on input and test results in XML and HTML on output.
The input can be class files stored in specified directory or in jar file (other file types then class are ignored).
Test results on output are in XML primarily. This information is then formatted in HTML using Xalan XSLT.

Data

Test results file is in XML format containing SigTestResults root element with following attributes:.

SigTestResults element can contain set of ElementDiff elements with following attributes:

ElementDiff element can contain set of AttributeDiff elements with following attributes:

 Signature test results XML file example:

<SigTestResults files="20" >
    <!-- "path-like" structure -->
    <tested> 
	<pathelement location="C:/netbeans/lib/patches/openide-compat.jar"/>
	<pathelement location="C:/netbeans/lib/openide.jar"/>
    </tested>
    <reference>
    	<pathelement location="C:/test/data/lib/openide.jar"/>
    </reference>

   <!-- Differences -->
   <ElementDiff package="aaa.bbb.ccc"
		 class-name="Abcd"
		 element_type="method"
		 element_name="doSomething"
		 change_type="add"
		 compatible="yes">

	<AttributeDiff name="parameters" test_value="Params par, File input" ref_value="String par" />
			...
    </ElementDiff>
</SigTestResults>

Test results are formatted in HTML page in following page report:

Configuration

The Signature tests configuration is basically on two levels:

1. Ant script level configuration
2. Test results report in HTML formatting

Ant script level configuration

Special defined ant task allows to create simple configurable Ant test scripts:

Smple signature test target element content:

<signaturetest output="../results/results.xml">
  <tested> <!-- "path-like" structure -->
    <pathelement location="${netbeans.home}/lib/patches/openide-compat.jar"/>
    <pathelement location="${netbeans.home}/lib/openide.jar"/>
  </tested>
  <reference>
    <pathelement location="../data/lib/openide.jar"/>
  </reference>
</signaturetest>

Test results report in HTML formatting

Test results in XML are formatted in  HTML using  XSL stylesheet. In XTest installation is predefined SigTestResults2html.xsl file in "xslt" directory. 

Processing

Ant test script invokes Signature test using special "signature_test" task. Signature test is going through the reference and test directory or jar file tree and each reference class type file compares with corresponding tested file. Comparison results are written in test results XML file. Following Ant task converts the XML file in HTML.

Error Conditions

All exceptions has to be handled and messages sent on standard output. 

Missing file in tested files set is detected as a "class element removed" change and compatibility is identified according rules defined above (changes compatibility). New class file not existing in reference files set is detected as "class element added"  compatible change.

Initialization and Shutdown

Siganture test is initialized by Ant and ends automatically when the task is finished.  

Unknowns

None

Constraints

None