A Practical Guide to Testing Object-Oriented Software

2017-07-07 02:10:07

The fundamental unit of an object-oriented program is a class. Class testing comprises those activities associated with verifying that the implementation of a class corresponds exactly with the specification for that class. If an implementation is correct, then each of the class's instances should behave properly.

Class testing is roughly analogous to unit testing in traditional testing processes and has many of the same problems that must be addressed (see sidebar). Class testing must also address some aspects of integration testing since each object defines a level of scope in which many methods interact around a set of instance attributes. Some of the most critical issues will be discussed in the context of concurrent issues in Chapter 8. The focus of this chapter is execution-based testing of classes. Our primary objective is to describe basic elements and strategies of testing classes, and we will focus on relatively simple classes. The testing of more complex classes will be addressed in the next two chapters.

We assume that a class to be tested has a complete and correct specification, and that it has been tested within the context of the models.^[1] We assume the specification is expressed in a specification language such as the Object Constraint Language (OCL) [WK99] or a natural language, and/or as a state transition diagram. If more than one form of specification is used for a class, we assume all forms are consistent and that information may be taken from whichever form is most useful as the basis for developing test cases for the class. We prefer to use the most formal specification for generating test cases.

^[1] Consistency is primarily a design consideration. When class testing is underway, design for the class should be finished at least as far as the current development iteration is concerned.

Ways to Test a Class

The code for a class can be tested effectively by review or by executing test cases. Review is a viable alternative to execution-based testing in some cases, but has two disadvantages over execution-based testing:

Reviews are subject to human error.

Reviews require considerably more effort with respect to regression testing, often requiring almost as many resources as the original testing.

While execution-based testing overcomes these disadvantages, considerable effort can be required for the identification of test cases and the development of test drivers. In some cases, the effort needed to construct a test driver for a class can exceed the effort of developing that class by several orders of magnitude. In that case, the costs and benefits of testing the class "outside" the system in which it will be used should be evaluated. This situation is not peculiar to object-oriented programming. The same situation arises in traditional procedural development with respect to many of the subprograms invoked at upper levels in a structure chart.

Traditional Unit Testing

The purpose of unit testing is to ensure that each unit meets its specification. If each unit meets its specification, then any bugs that appear when units are integrated together are more likely caused by incorrect interfacing of units than by incorrect implementations of the units. Debugging efforts can then be concentrated on the interfaces, not on the units themselves.

Unit testing is done as units are developed. In the procedural paradigm, a unit is a procedure (or function) or sometimes a group of procedures that implement an abstract data type. Units are typically tested by a combination of code inspections and execution testing, with most emphasis being placed on the latter. A simple unit test plan can be developed that identifies the test cases needed, and then a test driver can be constructed in a straightforward manner.

This all sounds good in theory, but in practice a number of stumbling blocks can arise. Typically, only the simplest of units those that appear as terminal nodes in a structure chart can be tested without significant effort. Test cases for such units tend to be easy to identify, and test drivers tend to be easy to construct if parameters do not have much structure to them.

Even units that have parameters with significant structure can sometimes be unit tested without significant effort if the driver can initialize the actual parameters with a relatively few assignment or read operations. Note, however, that this increases the amount of coupling between the unit and its test driver, which can increase maintenance costs if the structure changes over time.

While units at the lower levels in a structure chart can be unit tested in a straightforward way, at some point perhaps two or three levels from the bottom the interactions between units become so interwoven that unit testing becomes impractical. The effort required to produce a test driver can be greater than testing the unit in the context of testing a larger assembly. In some cases, the code for a test driver can be significantly larger than the code in the unit under test. This introduces an issue of unit testing the test driver itself.

Once we have identified executable test cases for a class, we must implement a test driver to run each of the test cases and report the results of each test case run. The test driver creates one or more instances of a class to run a test case. It is important to keep in mind that classes are tested by creating instances and testing the behavior of those instances (see Definitional versus Operational Semantics of Objects on page 19). The test driver can take a number of forms that we will describe later in this chapter. We favor the form of a separate "tester" class over the others because it offers a convenient organization for managing drivers and inheritance, and can be used to capture commonality among them. More benefits arise in testing class hierarchies, as we show in Chapter 7.

Dimensions of Class Testing

For each class, we must decide whether to test it independently as a unit or in some way as a component of a larger part of the system. We base that decision on the following factors:

The role of the class in the system, especially the degree of risk associated with it.

The complexity of the class measured in terms of the number of states, operations, and associations with other classes.

The amount of effort associated with developing a test driver for the class.

If a class is to be part of a class library, extensive class testing is appropriate even though the cost of developing a test driver might be high because its correct operation is essential. In the context of Brickles, we associate high risk with some of the most basic classes, such as Velocity and PuckSupply. If they are not implemented correctly, the game program will not work. Writing code to test these classes is straightforward because in the system design, they do not have to collaborate with other Brickles classes. We associate high risk with other classes, such as Puck, but we recognize these might not be easy to write a test driver for. They have associations in the design with many other classes, primarily because much of Puck's behavior is graphical. Puck associates with PlayField and any of the kinds of sprites in a playfield. We can foresee a significant effort in writing a test driver for Puck because all test cases will require an instance of a PlayField and some that are used for testing collision processing will require instances of Brick, BrickPile, and Paddle. Testing Puck, therefore, relies on an assumption that all these other classes work correctly. (We will examine this further in Chapter 6.) We might decide to test some or all of Puck in the context of cluster testing since we need instances of other classes to build environments around pucks suitable for testing them.

Let us now consider the five dimensions of testing in the context of testing a class.

Who

Classes are usually tested by its developer, just as subprograms traditionally are unit tested by their developer. Having a class developer also play the role of a class tester minimizes the number of people that have to understand a class's specification. It also facilitates implementation-based testing since the tester is intimately familiar with the code. Finally, the test driver can be used by the developer to debug the code as it is written.^[2]

^[2] A goal of testing is to find bugs, not to fix bugs. However, a useful component of class testing is in helping to isolate errors in the code.

The main disadvantage of test drivers and code being developed by the same person is that any misunderstandings of the specifications by the developer will be propagated to the test suite and test drivers. These potential problems are headed off by formal reviews of the code, and/or by requiring a test plan to be written by another class developer, and by allowing the code to be reviewed independently.

It is not unusual for independent testers to discover problems with the specifications for a class, so time should be allowed during testing to resolve them.

What

We primarily want to ensure that the code for a class exactly meets the requirements set forth in its specification. The amount of attention given to testing a class to ensure it does nothing more than what it is specified for depends on the risk associated with the class supplying extra behaviors. Incomplete coverage of code after a wide range of test cases have been run against the class could be an indication that the class contains extra, undocumented behaviors. Or it could merely suggest that the implementation must be tested using more test cases.

When

A test plan or at least some form of identification of test cases should be developed soon after a class is fully specified and ready for coding. This is particularly true when a class's developer is also responsible for its testing because early identification of test cases will help a developer to understand the specification and, as we mentioned, get feedback from an independent review. Take care when a class's developer is also responsible for its testing. A class developer who identifies incorrect or insufficient test cases will produce an implementation that passes all test cases, but that causes significant problems when the class is integrated into a larger part of a system.

Class testing can be done at various points in its development. In an incremental, iterative development process, the specification and/or the implementation for a class might need to be changed over the course of a project. Class testing should be performed prior to the use of the class in other portions of the software. Regression class testing should be performed whenever the implementation for a class has changed. If the changes resulted from the discovery of bugs in the code for the class, then a review of the test plan must be performed and test cases must be added or changed to detect those bugs during future testing.

How

Classes are usually tested by developing a test driver that creates instances of the class and sets up a suitable environment around those instances to run a test case. The driver sends one or more messages to an instance as specified by a test case, then checks the outcome of those messages based on a reply value, changes to the instance, and/or one or more of the parameters to the message. The test driver usually has responsibility for deleting any instances it creates if the language, such as C++, has programmer-managed storage allocation.

If a class has static data members and/or operations, then testing of those is required. These data members and methods belong to the class itself rather than to each instance of the class. The class can be treated as an object for example, in Java an instance of the class Class and tested according to what we describe in this chapter.

If the behavior of the instances of a class is based on the values of class-level attributes, then test cases for testing these class-level attributes must be considered as an extension of the state of the instances.

How Much

Adequacy can be measured in terms of how much of the specification and how much of the implementation has been tested. For class testing, we usually want to consider both. We want to test operations and state transitions in all sorts of combinations. Recall that objects maintain state and typically that state affects the meaning of operations. However, you need to consider whether exhaustive testing is feasible or even necessary. If not, then selective pair-wise combinations can be effective, especially when done in conjunction with risk analysis so that the most important test cases can be used and less important test cases can be sampled.