Extreme Programming Perspectives

Jester needs to know the JUnit test class (a subclass of TestCase that can be used by the JUnit TestRunners) and the directory that contains the source code that Jester is allowed to change.

The test class is the one that is expected to show any changes to code in the source directory. Typically, this test class would be the TestAll class of a package, and the source directory would be the subdirectory that contains the code being tested by that TestAll class.

This example use of Jester is for the Money samples of JUnit 3.2, which give a small example of how to use JUnit. There is an interface, IMoney, and two classes that implement the interface, Money and MoneyBag, and a test class, MoneyTest, which includes tests for both Money and MoneyBag. Including comments, there are about 400 lines of code.

Jester was invoked for this example as follows:

java jester.TestTester junit.samples.Money junit\samples

Jester made 47 separate modifications (including to the test class itself), ten of which did not make the tests fail; that is, they were changes that indicated possible missing tests or redundant code. This was a much higher percentage than expected. The version of Jester used ignores commented code. Because each modification requires recompilation and running all the tests, this took a long time to run, considering the amount of code. On a Pentium 133MHz (an old machine), Jester took 12 minutes to complete the run.

A description of the ten modifications that did not cause the tests to fail follows. Three of the modifications were in the equals method of the Money class:

public boolean equals(Object anObject) { if (isNull()) if (anObject instanceof IMoney) return ((IMoney)anObject).isNull(); ...

Jester reported that:

• if (isNull()) can be replaced by if (false && isNull()) • if (anObject instanceof IMoney) can be replaced by if (true || anObject instanceof IMoney) • if (anObject instanceof IMoney) can be replaced by if (false && anObject instanceof IMoney)

The first of these shows that either isNull() is always false in the tests (thus a test is missing for the case where isNull() is true), or it could show that it makes no difference to the running of the tests whether the isNull() branch is executed. In fact, there is no test of equals for a "null" Money. (The method isNull has been renamed isZero in JUnit 3.4). Without further examination of the code, the possibility that the branch of code does not make any difference to the correct running of code should not be discounted. This could happen, for example, if the isNull() branch was a behavior-neutral optimization.

The second and third modifications reported indicate that either that if statement is not executed or it makes no difference to the running of the code that is, it doesn't matter whether the value of the condition is true or false. In this case, this code is not executed by the tests. A conventional code coverage tool would be able to spot this.

The other seven modifications that Jester made that did not cause the tests to fail were all for the MoneyBag class. One of these was changing the construction of a vector from new Vector(5) to new Vector(6). This had no effect on the correct running of the code, because the effect of this number is on the initial internal size of the constructed vector, which can have an effect on performance but does not affect the vector's behavior. Another change was to modify the hashCode value of an empty MoneyBag. This has no effect on the correct running of the code and can be considered a "false hit" by Jester.

Three of the other modifications are similar to those for Money; they show that the equals method is not tested for "null" MoneyBags.

The remaining two modifications both relate to the equals method for the special case that two MoneyBags that contain a different number of Money objects are not equals. There are no unit tests for this special case code.

Jester has also been applied to parts of Sidewize, a browser companion built by Connextra.^[3]It successfully identified where tests were missing and where code had become redundant and needed removing. However, it took considerable analysis of the results to identify whether the modifications reported by Jester represented missing tests or redundant code, or were simply false hits that is, represented behavior preserving changes to the code.

^[3] See http://www.sidewize.com.