Use Case Testing

The Insect Keeper General, sitting astride his giant hovering aphid, surveyed the battlefield which reeked with the stench of decay and resonated with the low drone of the tattered and dying mutant swarms as their legs kicked forlornly at the sky before turning to his master and saying, 'My Lord, your flies are undone.'

— Andrew Vincent

Introduction

Up until now we have examined test case design techniques for parts of a system—input variables with their ranges and boundaries, business rules as represented in decision tables, and system behaviors as represented in state-transition diagrams. Now it is time to consider test cases that exercise a system's functionalities from start to finish by testing each of its individual transactions.

Defining the transactions that a system processes is a vital part of the requirements definition process. Various approaches to documenting these transactions have been used in the past. Examples include flowcharts, HIPO diagrams, and text. Today, the most popular approach is the use case diagram. Like decision tables and state-transition diagrams, use cases are usually created by developers for developers. But, like these other techniques, use cases hold a wealth of information useful to testers.

Use cases were created by Ivar Jacobsen and popularized in his book Object-Oriented Software Engineering: A Use Case Driven Approach. Jacobsen defines a "use case" as a scenario that describes the use of a system by an actor to accomplish a specific goal. By "actor" we mean a user, playing a role with respect to the system, seeking to use the system to accomplish something worthwhile within a particular context. Actors are generally people although other systems may also be actors. A "scenario" is a sequence of steps that describe the interactions between the actor and the system. Note that the use case is defined from the perspective of the user, not the system. Note also that the internal workings of the system, while vital, are not part of the use case definition. The set of use cases makes up the functional requirements of a system.

The Unified Modeling Language notion for use cases is:

Figure 9-1: Some Stateless University use cases.

The stick figures represent the actors, the ellipses represent the use cases, and the arrows show which actors initiate which use cases.

It is important to note that while use cases were created in the context of object-oriented systems development, they are equally useful in defining functional requirements in other development paradigms as well.

The value of use cases is that they:

• Capture the system's functional requirements from the user's perspective; not from a technical perspective, and irrespective of the development paradigm to be used.
• Can be used to actively involve users in the requirements gathering and definition process.
• Provide the basis for identifying a system's key internal components, structures, databases, and relationships.
• Serve as the foundation for developing test cases at the system and acceptance level.

Technique

Unfortunately, the level of detail specified in the use cases is not sufficient, either for developers or testers. In his book Writing Effective Use Cases, Alistair Cockburn has proposed a detailed template for describing use cases. The following is adapted from his work:

Example

Consider the following example from the Stateless University Registration System. A student wants to register for a course using SU's online registration system, SURS.

Hopefully each use case has been through an inspection process before it was implemented. To test the implementation, the basic rule is to create at least one test case for the main success scenario and at least one test case for each extension.

Because use cases do not specify input data, the tester must select it. Typically we use the equivalence class and boundary value techniques described earlier. Also a Domain Test Matrix (see the Domain Analysis Testing chapter for an example) may be a useful way of documenting the test cases.

It is important to consider the risk of the transaction and its variants under test. Less risky transactions merit less testing. More risky transactions should receive more testing. For them consider the following approach.

To create test cases, start with normal data for the most often used transactions. Then move to boundary values and invalid data. Next, choose transactions that, while not used often, are vital to the success of the system (i.e., Shut Down The Nuclear Reactor). Make sure you have at least one test case for every Extension in the use case. Try transactions in strange orders. Violate the preconditions (if that can happen in actual use). If a transaction has loops, don't just loop through once or twice—be diabolical. Look for the longest, most convoluted path through the transaction and try it. If transactions should be executed in some logical order, try a different order. Instead of entering data top-down, try bottom-up. Create "goofy" test cases. If you don't try strange things, you know the users will.

Free Stuff

Download Holodeck from http://www.sisecure.com/holodeck/holodeck-trial.aspx.

Most paths through a transaction are easy to create. They correspond to valid and invalid data being entered. More difficult are those paths due to some kind of exceptional condition—low memory, disk full, connection lost, driver not loaded, etc. It can be very time consuming for the tester to create or simulate these conditions. Fortunately, a tool is available to help the tester simulate these problems—Holodeck, created by James Whittaker and his associates at Florida Institute of Technology. Holodeck monitors the interactions between an application and its operating system. It logs each system call and enables the tester to simulate a failure of any call at will. In this way, the disk can be "made full," network connections can "become disconnected," data transmission can "be garbled," and a host of other problems can be simulated.

A major component of transaction testing is test data. Boris Beizer suggests that 30 percent to 40 percent of the effort in transaction testing is generating, capturing, or extracting test data. Don't forget to include resources (time and people) for this work in your project's budget.

Applicability and Limitations

Transaction testing is generally the cornerstone of system and acceptance testing. It should be used whenever system transactions are well defined. If system transactions are not well defined, you might consider polishing up your resume or C.V.

While creating at least one test case for the main success scenario and at least one for each extension provides some level of test coverage, it is clear that, no matter how much we try, most input combinations will remain untested. Do not be overconfident about the quality of the system at this point.

Summary

• A use case is a scenario that describes the use of a system by an actor to accomplish a specific goal. An "actor" is a user, playing a role with respect to the system, seeking to use the system to accomplish something worthwhile within a particular context. A scenario is a sequence of steps that describe the interactions between the actor and the system.
• A major component of transaction testing is test data. Boris Beizer suggests that 30 percent to 40 percent of the effort in transaction testing is generating, capturing, or extracting test data. Don't forget to include resources (time and people) for this work in your project's budget.
• While creating at least one test case for the main success scenario and at least one for each extension provides some level of test coverage, it is clear that, no matter how much we try, most input combinations will remain untested. Do not be overconfident about the quality of the system at this point.

Practice

1. Given the "Register For A Course" use case for the Stateless University Registration System described previously, create a set of test cases so that the main success scenario and each of the extensions are tested at least once. Choose "interesting" test data using the equivalence class and boundary value techniques.

References

Beizer, Boris (1990). Software Testing Techniques (Second Edition). Van Nostrand Reinhold.

Beizer, Boris (1995). Black-Box Testing: Techniques for Functional Testing of Software and Systems. John Wiley & Sons.

Cockburn, Alistair (2000). Writing Effective Use Cases. Addison-Wesley.

Fowler, Martin and Kendall Scott (1999). UML Distilled: A Brief Guide to the Standard Object Modeling Language (2nd Edition). Addison-Wesley.

Jacobsen, Ivar, et al. (1992). Object-Oriented Systems Engineering: A Use Case Driven Approach. Addison-Wesley.