Boundary Value Testing

The Prince looked down at the motionless form of Sleeping Beauty, wondering how her supple lips would feel against his own and contemplating whether or not an Altoid was strong enough to stand up against the kind of morning breath only a hundred years' nap could create.

— Lynne Sella

Introduction

Equivalence class testing is the most basic test design technique. It helps testers choose a small subset of possible test cases while maintaining reasonable coverage. Equivalence class testing has a second benefit. It leads us to the idea of boundary value testing, the second key test design technique to be presented.

In the previous chapter the following rules were given that indicate how we should process employment applications based on a person's age. The rules were:

Notice the problem at the boundaries—the "edges" of each class. The age "16" is included in two different equivalence classes (as are 18 and 55). The first rule says don't hire a 16-year-old. The second rule says a 16-year-old can be hired on a part-time basis.

Boundary value testing focuses on the boundaries simply because that is where so many defects hide. Experienced testers have encountered this situation many times. Inexperienced testers may have an intuitive feel that mistakes will occur most often at the boundaries. These defects can be in the requirements (as shown above) or in the code as shown below:

If (applicantAge >= 0 && applicantAge <=16) hireStatus="NO"; If (applicantAge >= 16 && applicantAge <=18) hireStatus="PART"; If (applicantAge >= 18 && applicantAge <=55) hireStatus="FULL"; If (applicantAge >= 55 && applicantAge <=99) hireStatus="NO";

Of course, the mistake that programmers make is coding inequality tests improperly. Writing > (greater than) instead of ≥ (greater than or equal) is an example.

The most efficient way of finding such defects, either in the requirements or the code, is through inspection. Gilb and Graham's book, Software Inspection, is an excellent guide to this process. However, no matter how effective our inspections, we will want to test the code to verify its correctness.

Perhaps this is what our organization meant:

What about ages -3 and 101? Note that the requirements do not specify how these values should be treated. We could guess but "guessing the requirements" is not an acceptable practice.

The code that implements the corrected rules is:

If (applicantAge >= 0 && applicantAge <=15) hireStatus="NO"; If (applicantAge >= 16 && applicantAge <=17) hireStatus="PART"; If (applicantAge >= 18 && applicantAge <=54) hireStatus="FULL"; If (applicantAge >= 55 && applicantAge <=99) hireStatus="NO";

The interesting values on or near the boundaries in this example are {-1, 0, 1}, {15, 16, 17}, {17, 18, 19}, {54, 55, 56}, and {98, 99, 100}. Other values, such as {-42, 1001, FRED, %$#@} might be included depending on the module's documented preconditions.

Technique

The steps for using boundary value testing are simple. First, identify the equivalence classes. Second, identify the boundaries of each equivalence class. Third, create test cases for each boundary value by choosing one point on the boundary, one point just below the boundary, and one point just above the boundary. "Below" and "above" are relative terms and depend on the data value's units. If the boundary is 16 and the unit is "integer" then the "below" point is 15 and the "above" point is 17. If the boundary is $5.00 and the unit is "US dollars and cents" then the below point is $4.99 and the above point is $5.01. On the other hand, if the value is $5 and the unit is "US dollars" then the below point is $4 and the above point is $6.

Note that a point just above one boundary may be in another equivalence class. There is no reason to duplicate the test. The same may be true of the point just below the boundary.

You could, of course, create additional test cases farther from the boundaries (within equivalence classes) if you have the resources. As discussed in the previous chapter, these additional test cases may make you feel warm and fuzzy, but they rarely discover additional defects.

Boundary value testing is most appropriate where the input is a continuous range of values. Returning again to the Goofy Mortgage Company, what are the interesting boundary values? For monthly income the boundaries are $1,000/month and $83,333/month (assuming the units to be US dollars).

Figure 4-1: Boundary values for a continuous range of inputs.

Test data input of {$999, $1,000, $1,001} on the low end and {$83,332, $83,333, $83,334} on the high end are chosen to test the boundaries.

Because GMC will write a mortgage for one through five houses, zero or fewer houses is not a legitimate input nor is six or greater. These identify the boundaries for testing.

Figure 4-2: Boundary values for a discrete range of inputs.

Rarely will we have the time to create individual tests for every boundary value of every input value that enters our system. More often, we will create test cases that test a number of input fields simultaneously.

Plotting "monthly income" on the x-axis and "number of dwellings" on the y-axis shows the "locations" of the test data points.

Figure 4-3: Data points on the boundaries and data points just outside the boundaries.

Note that four of the input combinations are on the boundaries while eight are just outside. Also note that the points outside always combine one valid value with one invalid value (just one unit lower or one unit higher).

Examples

Boundary value testing is applicable to the structure (length and character type) of input data as well as its value. Consider the following two examples:

Example 1

Referring to the Trade Web page of the Brown & Donaldson Web site described in Appendix A, consider the Quantity field. Input to this field can be between one and four numeric characters (0,1, ..., 8,9). A set of boundary value test cases for the length attribute would be {0, 1, 4, 5} numeric characters.

Example 2

Again, on the Trade page, consider the Quantity field, but this time for value rather than structure (length and character type). Whether the transaction is Buy or Sell, the minimum legitimate value is 1 so use {0, 1, 2} for boundary testing. The upper limit on this field's value is more complicated. If the transaction is Sell, what is the maximum number of shares that can be sold? It is the number currently owned. For this boundary use {sharesOwned-1, sharesOwned, sharesOwned+1}. If the transaction is Buy, the maximum value (number of shares to be purchased) is defined as

shares = (accountBalance - commission) / sharePrice

assuming a fixed commission. Use {shares-1, shares, shares+1} as the boundary value test cases.

Applicability and Limitations

Boundary value testing can significantly reduce the number of test cases that must be created and executed. It is most suited to systems in which much of the input data takes on values within ranges or within sets.

Boundary value testing is equally applicable at the unit, integration, system, and acceptance test levels. All it requires are inputs that can be partitioned and boundaries that can be identified based on the system's requirements.

Summary

• While equivalence class testing is useful, its greatest contribution is to lead us to boundary value testing.
• Boundary value testing is a technique used to reduce the number of test cases to a manageable size while still maintaining reasonable coverage.
• Boundary value testing focuses on the boundaries because that is where so many defects hide. Experienced testers have encountered this situation many times. Inexperienced testers may have an intuitive feel that mistakes will occur most often at the boundaries.
• Create test cases for each boundary value by choosing one point on the boundary, one point just below the boundary, and one point just above the boundary. "Below" and "above" are relative terms and depend on the data value's units.

Practice

1. The following exercises refer to the Stateless University Registration System Web site described in Appendix B. Define the boundaries, and suitable boundary value test cases for the following:
   1. ZIP Code—five numeric digits.
   2. First consider ZIP Code just in terms of digits. Then, determine the lowest and highest legitimate ZIP Codes in the United States. For extra credit [1], determine the format of postal codes for Canada and the lowest and highest valid values.
   3. Last Name—one through fifteen characters (including alphabetic characters, periods, hyphens, apostrophes, spaces, and numbers). For extra credit [2] create a few very complex Last Names. Can you determine the "rules" for legitimate Last Names? For additional extra credit [3] use a phonebook from another country—try Finland or Thailand.
   4. User ID—eight characters at least two of which are not alphabetic (numeric, special, nonprinting).
   5. Course ID—three alpha characters representing the department followed by a six-digit integer which is the unique course identification number. The possible departments are:
      • PHY - Physics
      • EGR - Engineering
      • ENG - English
      • LAN - Foreign languages
      • CHM - Chemistry
      • MAT - Mathematics
      • PED - Physical education
      • SOC - Sociology

[1]There actually is no extra credit, so do it for fun.

[2]There actually is no extra credit, so do it for fun.

[3]There actually is no extra credit, so do it for fun.

References

Beizer, Boris (1990). Software Testing Techniques. Van Nostrand Reinhold.

Gilb, Tom and Dorothy Graham (1993). Software Inspection. Addison-Wesley. ISBN 0-201-63181-4.

Myers, Glenford J. (1979). The Art of Software Testing. John Wiley & Sons.