A Practical Guide to Testing Object-Oriented Software

Implementing classes is more straightforward when done from the top of the hierarchy down. In the same way, testing classes in an inheritance hierarchy is generally more straightforward when approached from the top down. In testing first at the top of a hierarchy, we can address the common interface and code and then specialize the test driver code for each subclass. Implementing inheritance hierarchies from the bottom up can require significant refactoring of common code into a new superclass. The same thing can happen to test drivers. To keep our discussion simpler, we will assume that the classes in an inheritance hierarchy are to be tested top down. First, we will focus on testing a subclass of a class that has already been tested.

Consider that we would like to test a class D that is a subclass of another class C. Assume C has already been tested adequately by the execution of test cases by a test driver. What do we need to test in D?

Since D inherits at least part of its specification and also part of its implementation from C, it seems reasonable to assume that some of the test software for C can be reused in testing D. That is indeed the case. Consider, for example, the degenerate case in which D inherits from C and makes no changes at all. Thus, D is equivalent to C in its specification and implementation. Class D need not be tested at all if we are willing to assume that the compiler correctly processes the code. Under such an assumption, if C passes all its test cases, then so must D.

In the more general case in which D contains incremental changes from C, the effort needed to test D adequately can be reduced by reusing test cases and parts of the test driver for C. We will show how we can extend the testing done for C in a straightforward way to test D.

Refinement Possibilities

As supported by Java and C++, inheritance permits only a small number of incremental changes in deriving a class D from a class C. We can define a new derived class D that differs from C in only four general ways:

1. Add one or more new operations in the interface of D and possibly a new method in D to implement each new operation.^[2]

   ^[2] A new operation might be abstract ( pure virtual in C++ terminology), deferring implementation to subclasses.

2. Change the specification or implementation of an operation declared by C in one or two ways:

   1. Change in D the specification for an operation declared in C.

   2. Override in D a method^[3] in C that implements an operation inherited by D.

      ^[3] We assume if the class is implemented in C++, then such operations are declared virtual in the base class. Failure to use a virtual member function violates the substitution principle.

   Note that either or both of these can apply. It is common to override a method in a subclass. It is also possible to change a specification for an operation without directly changing the method that implements the operation in a subclass.^[4]

   ^[4] For example, the implementation might be based on a Template Method pattern [GHJV94].

3. Add into D one or more new instance variables to implement more states and/or attributes.

4. Change the class invariant in D.

While inheritance can be used for many reasons, we will assume that inheritance is used only in accordance with the substitution principle. This is a reasonable assumption because many of the benefits of object-oriented programming arise from polymorphism. The substitution principle ensures that objects bound to an interface behave as expected, thereby resulting in more reliable and readable code. We also assume that the principle of information hiding is followed so that any data in an object is not public. If data is indeed public, then we will augment our discussion with an assumption that reads and writes to public data correspond to implicit get and set operations,respectively.

Since D inherits part of its specification from C, then all the specification-based test cases used in testing C can be used in testing D. The substitution principle ensures that all the test cases still apply. We need new, additional specification-based test cases for new operations, perhaps additional specification-based test cases for operations whose preconditions have been weakened or postconditions have been strengthened, and implementation-based test cases to test new methods. If the class invariant has been refined in the subclass, then we will need to add test cases to address the refinements.

Hierarchical, Incremental Testing

The incremental changes between class C and its derived class D can be used to guide the identification of what needs to be tested in D. Consider the incremental changes from a testing perspective. Since D is a subtype of C, then all the specification-based test cases for C also apply to D. Many of the implementation-based and interaction-based test cases also apply. We use the term inherited test cases to refer to the test cases for a subclass that were identified for testing its base class. We can determine which inherited test cases apply to testing a subclass through a straightforward analysis. As part of that same analysis, we can determine which inherited test cases do not have to be executed in testing the subclass. We repeat here the list of incremental changes given in the previous section and examine each from the testing perspective.

1. Add one or more new operations in the interface of D and possibly a new method in D to implement each new operation.

   A new operation introduces new functionality and new code to test. A new operation does not directly affect existing, inherited operations or methods. We need to add specification-based test cases for each new operation. We need to add implementation-based and interaction-based test cases in order to comply with coverage criteria in the test plan if the operation is not abstract and has an implementation.

2. Change the specification or implementation of an operation declared by C in one or two ways:

   1. Change in D the specification for an operation declared in C.

      We need to add new specification-based test cases for the operation. Additional test cases provide new inputs that meet any weakened preconditions and check outputs for the new expected results that result from any strengthened postconditions. The test cases for this operation defined for C still apply, but must be re-run. In addition, we need to add strengthened postcondition requirements to the output for each of the test cases used to test this operation in class C.

   2. Override in D a method in C that implements an operation inherited by D.

      We can reuse all the inherited specification-based test cases for the method. Since there is new code to test, we will need to review each of the implementation-based test cases and interaction-based test cases, revising and adding to them as needed to meet the test criteria for coverage.

3. Add into D one or more new instance variables to implement more states and/or attributes.

   A new variable is added most likely in connection with new operations and/or code in overriding methods, and testing will be handled in connection with them. If a new variable is not used in any method, then we do not have to make any changes.

4. Change the class invariant in D.

   Class invariants amount to additional postconditions for every test case. We prefer to view them as implied postconditions and to write test cases without explicit references to invariant constraints. Test case output is subject to invariant constraints that is, "and the class invariant holds" is implicit in every test case output. Thus, if a class invariant changes, then we need to rerun all inherited test cases to verify that the new invariant holds.

We do not need to add specification-based test cases for operations that are unchanged from base class to derived class. The test cases can be reused as is. We do not need to run any of these test cases if the operations they test have not changed in any way that is, in specification or in implementation. We do, however, need to rerun any test cases for an operation if its method has changed indirectly because it uses an operation that itself has changed. We also might need additional implementation-based test cases for such methods.

We refer to applying the above analysis and its results as hierarchical incremental testing (HIT). We can use the analysis to determine for a subclass what test cases need to be added, what inherited test cases need to be run, and what inherited test cases do not need to be run. Determining which test cases do not need to be run is a bit tricky. In practice, it is usually easier and more reliable to just rerun all test cases. However, it pays to determine which test cases can be reused.

Figure 7.2 summarizes the analysis associated with HIT. We classify each operation defined for a derived class D in the first column as new, refined, and unchanged. The second column specifies whether that change affects specification-based testing. The third column specifies whether that change affects implementation-based testing. The table adds a dimension of public and private. Private features of a class do not affect the public interface.

A No entry in the table indicates that the incremental change (for the row containing the entry) has no incremental effect on the test suite that is, the test cases for the superclass are still valid for the subclass. A Yes entry indicates that test cases must be added to address that incremental change. A Maybe entry indicates that a tester must examine the code in the implementation to determine if more test cases are needed to achieve some level of coverage. As a short example, consider the Timer class in the design of Brickles that represents the passing of time as a sequence of discrete "ticks." Each timer event is processed by a Timer instance that notifies other objects in the match. Those objects, in turn, process another timer tick. This aspect of the design is based on the Observer pattern [GHJV94]. A Timer instance occupies the role of subject, and other objects in a Brickles match assume the role of observers. If we implement the design as shown in Figure 7.3 based on existing, tested classes Subject and Observer prescribed by the Observer pattern, then we can identify from Figure 7.2 what needs to be tested in class Timer. Specifically, the specifications of attach(), detach(), and notify() do not need to be tested further because their specifications have not changed from what was defined in Subject. No new implementation-based test cases are needed either because the code (not shown) reveals that there has been no changes to the execution flow in these methods that is, these methods do not use any of the new code in Timer or there is no new interactions with other objects. We do need to add specification-based test cases for the new operation tick(), which processes a timer event, and implementation-based test cases for the method that implements it. With respect to testing TimerObserver, HIT shows we need to test the overridden update() operation by adding specification-based test cases because the specification of the operation changes in the subclass (it specifies possible state changes in a concrete subclass). Since the operation is abstract in Observer, the Maybe in the HIT table translates to a need to add implementation-based test cases as well.

Let us now examine hierarchical incremental testing from the context of a test plan that is, from the perspective of identifying test cases. Then we will examine it from a more detailed level. We will use as an example the inheritance hierarchy rooted at Sprite in Brickles (see Figure 7.4). A sprite is an abstraction that represents any object that can appear on a playfield in anarcade game. The name has historic significance in the domain of arcade games [Hens96]. Some attributes associated with a sprite are a bitmap that renders a visual image, a size that describes the width and height of the bitmap, a location on a playfield, and a bounding rectangle that is the smallest rectangular area of the playfield that contains the sprite's image (if it is on a playfield^[5]).

^[5] Consider, for example a puck in play and a puck not yet put into play. The former is on a playfield, the latter is not.

A movable sprite is a sprite that can change position in a playfield. Associated with a movable sprite is the velocity at which it is currently moving. A velocity represents a direction and a distance traveled (in playfield units) in a unit of time. In our model, a puck and a paddle are both concrete kinds of movable sprites.

A stationary sprite is a sprite whose position is fixed as long as it is on the playfield. In our model, a brick is an example of a stationary sprite. Since we have only one kind of stationary sprite in Brickles, we have probably shortsightedly elected not to represent stationary sprites by an abstract class in the current increment. Thus, class Brick inherits directly from Sprite in our model.

Specifications for some of the operations in these classes in the Sprite hierarchy are given in Figure 7.5.

Specification-Based Test Cases

Under hierarchical, incremental testing, changes in a subclass's specification from the specification of its base class determine what needs to be tested. Test requirements are summarized in the column labeled Affect Class Specification? in Figure 7.2. While our discussion will be based on the relatively informal specifications given in Figure 7.5, the techniques apply to any form of specification, including Object Constraint Language (OCL) and state transition diagrams.

Let us focus first on the class MovableSprite, assuming test cases have been identified and implemented for class Sprite (see Figure 7.6).^[6] MovableSprite adds some new operations and attributes to model motion in a playfield and also overrides some methods. Among the new operations in class MovableSprite are move(), which updates a movable sprite's position in a playfield; setVelocity(const Velocity &), which changes the velocity at which a movable sprite is moving; isMoving() const, which inspects whether a movable sprite is currently in a moving state; and collideInto(Sprite &), which modifies the state of a movable sprite to reflect a collision with some other sprite in the playfield. Among the overridden methods are the constructor. Most of the operations declared by Sprite are inherited unchanged.

^[6] We'll address the problem of testing an abstract class, such as Sprite, later in this chapter.

The implementation of MovableSprite uses the following two new variables to store the velocity attribute and indicate whether the sprite is moving:

• _currentVelocity, which is an instance of class Velocity used to store the current velocity.

• _isMoving, which indicates whether the movable sprite is currently in motion. When this variable is false, move() has no effect.

What do we need to do to adequately test MovableSprite given that Sprite has already been tested? The subclass's code is based on the code tested in the superclass. From the class model (Figure 7.4) and the specifications for the operations shown in Figure 7.5, we can identify the following based on the HIT information in Figure 7.2:

• The changed invariant demands that all the test cases defined for Sprite should be run for MovableSprite and the new invariant checked.

• The new operations in MovableSprite need to have specification-based test cases generated as well as implementation-based test cases generated. We will want to check interactions among many of the new operations for example, setting the velocity and then moving a movable sprite a few times to ensure it has adopted the specified velocity, or changing the velocity and verifying that the heading (up, down, left, or right) is correct.

• The operations in Sprite for which methods have not been overridden in MovableSprite need no additional test cases.

Implementation-Based Test Cases

The column labeled Affect Class Implementation? in Figure 7.2 specifies what needs to be tested with respect to implementation. If an entry contains Maybe, then a tester must examine the code to determine whether additional test cases are required. In the case of MovableSprite, quite a few methods have been added to implement the operations concerned with movement. Methods for operations associated with a position in the playfield have not been overridden. The method tick() is overridden so that it causes a movable sprite to change position in the playfield based on its current velocity.

Based on the information in Figure 7.2, we can determine the following about implementation-based testing of MovableSprite:

• No new test cases are needed for size(), bitmap(), boundingRect(), overlaps(), position(), setPosition(), or playField(). After examining the code for these methods and determining that there are no interactions among them with tick(), we conclude these test cases do not need to be rerun.

• Implementation-based test cases are needed for all the new methods such as reverse(), move(), and so on.

• Implementation-based test cases are needed for the implementation of the abstract method tick().

We also need interaction test cases associated with checking the correct implementation of startMoving() and stopMoving() and the effect of the state change on other operations such as tick() and reverse().