Some Final Thoughts…

Overview

"Once you eliminate your number-one problem, number two gets a promotion."

— Rudy's Rutabaga Rule

Gerald M. Weinberg's Secrets of Consulting

After reading a book like this or attending a seminar or training session, you no doubt ask yourself, "What now?" Hopefully you realize that very few readers of this book need to do everything that we've suggested. Similarly, the implementation of each topic in this book will produce a different ROI than every other topic. The number-one priority for one organization might not necessarily be as important to other organizations.

We feel that models such as TPI (refer to Chapter 11) can help most organizations choose the next logical step in the quest for process improvement. Some of you, though, are still not satisfied. You want to know what you can do now. Today! Naturally, we can't answer that question without knowing a lot more about your organization's processes, skills, budget, politics, quality of software produced, and a multitude of other things. There is, though, a short list of things that we think have a high return on investment for almost every organization. These are the things that we believe every testing organization should strive to do:

• Use Preventive Testing Techniques
• Conduct Software Risk Analysis
• Determine Planning Risks
• Develop a Testing Strategy
• Use Inventories
• Use Testing Tools When Appropriate
• Analyze Defect Trends and Patterns
• Measure Test Effectiveness
• Conduct Training Continually
• Sell the Idea of Testing

Use Preventive Testing Techniques

There's nothing a testing organization can do that will provide a greater return on investment than actually preventing defects or finding them very early in the software development lifecycle. Chapter 1 describes the concept of preventive testing, and the rest of this book is based upon the concept that testing activities will occur in advance of or parallel to the software development.

Conduct Software Risk Analysis

It's a fact of life. It's just not possible to test everything, so what you test is much more important than how much you test. Analyzing software risks can help testing organizations prioritize their tests and determine where to focus their testing effort.

Determine Planning Risks

We've never taken part in a project where everything went according to the original plan. Requirements change, testers get sick, the code is late, and the list goes on. Since we can't create a perfect plan, we have to identify potential planning risks and formulate viable contingencies. These contingencies will always involve one or more of the following actions: adding resources, changing the schedule, reducing the scope of the project, or reducing some quality activities (like testing). The key is to decide and agree upon the best contingency for each planning risk in advance.

Develop a Testing Strategy

There are many issues that need to be considered when planning a testing effort. Who should do the testing? When should we start? How will we know when we're done? What should be automated? Do we need training? What metrics are required to measure the effectiveness of our effort?

Test planning helps address questions like these and helps in obtaining buy-in from key parties by involving them in the test planning early in the development lifecycle.

Use Inventories

Many organizations struggle to determine what actual conditions should be tested. Inventories are one way of analyzing a product in order to develop lists of potential test conditions. The inventory tracking matrix that is a by-product of developing inventories facilitates the maintenance of the test set and provides a method of determining coverage. When coupled with software risk analysis (see above), inventories can provide the basis for a sound testing strategy.

Use Testing Tools When Appropriate

Testing tools are not a panacea and are certainly not the answer to all the problems that test teams might encounter, but there are many times when the appropriate use of a tool can help the test team be more effective or productive. You may or may not, in fact, choose to use tools on any given project, but their use should always be considered when creating your testing strategy. When trying to decide whether or not to use tools, remember to consider the pitfalls we discussed in Chapter 6.

Analyze Defect Trends and Patterns

There is a great deal that one can learn by analyzing the trends, patterns, type, source, severity, age, etc. of defects discovered and missed by the testers. Specifically, testers can identify process improvement and training opportunities and discover risky components that warrant a greater dose of testing.

Measure Test Effectiveness

It's very difficult for the manager of a testing team to determine how well his or her team is doing if the manager does not have a way to measure test effectiveness. Chapter 7 outlines several different metrics for measuring test effectiveness. We believe that every test manager should measure coverage and use at least one other measure of test effectiveness such as Defect Removal Efficiency (DRE).

Conduct Training Continually

Testing is now recognized by most software engineers as a discipline. In testing, as in every discipline, there's a body of knowledge and certain skills that every professional should possess. Additionally, some testers will require special or advanced skills or knowledge based upon the kind of testing they perform.

Chapter 10 lists many of the types of training that are available and explains some of the pros and cons of each. Remember that training needs to be timely, appropriate, and ongoing.

Sell the Idea of Testing

There are still a lot of "nay sayers" in this world who are not truly sold on the benefits of testing. It's up to the test manager and test teams to sell developers, users, and management on the value of testing. Testers need to make effective use of metrics, pilots, and public forums to advance their cause.

"It ain't over till it's over…."

- Yogi Berra