Design for Trustworthy Software: Tools, Techniques, and Methodology of Developing Robust Software

The triad of design methods described in Figure 12.3 (QFD, TRIZ, Taguchi Methods) is about as far upstream as you can get in the design and development of a new product or process. But perhaps one matter is still logically priorconcept selection (what needs to be invented). The customer more often talks in terms of desiderata or vague needs. He or she is not an engineer and often cannot provide clear, unambiguous input for the QFD process. The Scottish engineering design professor Stuart Pugh has developed a formal design concept selection method that we will feature here as the preferred method in its context as a utility function method. Design concept selection, although logically prior to QFD, does not strictly precede the QFD process. Instead, it's contemporaneous. It comes before, during, and after, because it is highly iterative. Concept selection is picking the idea (actually, ideas) that best satisfy the product requirements as they emerge from QFD and the voice of the customer. Concept selection is the first attempt (or attempts, actually) to map the user's needs to a viable software architecture. Durfee argues that concept selection implies that the design team in consort with the customer actually weeds out bad ideas rather than trying to select the best one.^[11] He also notes that concept must be a structured process:

• Customer needs drive the process.

• All criteria are considered.

• Choices are documented.

• Choices can be defended later in the design cycle.

• The process should be immune from the influence of a champion.

The principal steps of the Pugh process more precisely define a structured process as follows:^[12]

We will return to the Pugh method shortly. First, to set it in context, we consider the general utility function method. The utility, or total effectiveness, of each design alternative is obtained by summing the numerical scores assigned to the individual evaluation criteria weighted for its importance to the final result.^[13] The utility equation is

U_i = Σ w_i x_ij (j = 1, 2, 3,..., m)

U_i is the utility of the jth design concept alternative. w_i is the weighting coefficient or measure of importance for the ith evaluation criterion, x_i. m is the number of concept alternatives considered. The sum is from 1 to n, the number of criteria.

Utility function analysis is used in cases where it is necessary to evaluate both quantitative and qualitative considerations at the same time. The evaluation criteria are derived from customer requirements, engineering specifications, and insights developed during concept generation. It is important to have a reasonable number of clear, independent criteria to support a good design decision. Because not all criteria are equally important, it is necessary to weight them for relative importance. Normally the sum of the weights is normalized to 1.0. The various candidate criteria are usually scored from 1 to 5. However, if more resolution is needed, and the data can support it, a scale of 1 to 10 may be used. In many cases you may use a reference scoring method whereby a best in class or standard design of some sort is chosen as a reference and is given a score of 3 out of 5 or 2 out of 3. Surprisingly, the utility scores for the various methods usually show clear ranking and separation. Everything else is then the same as, better than, or worse than the reference. If two or three design alternatives are closely ranked in utility, they may all be continued into the next design phase if the budget allows. Utility function analysis is quite amenable to computer spreadsheet analysis. This allows the design team to vary weights and criteria scores to achieve separation between utility values until a clear concept winner emerges. Stuart Pugh's method is a simplified version of the utility function method. Pugh's simplifications may be summarized as follows:, ^[14]

• All evaluation criteria are assumed to be of equal importance.

• The alternatives are scored using the reference-based scoring method.

• Instead of a point scale, alternatives are scored with respect to the reference as better than, the same as, or worse than.

• The overall score is determined by counting the better, same, and worse values for each alternative.

Pugh's method facilitates concept selection in several ways:

• Scoring is simple. Deciding on whether something is the same as, better than, or worse than some characteristic is much simpler than assigning it a numeric value.

• The method is easy to use and adapt as the team gains insight. It is more repeatable than numeric schemes.

• The matrix or spreadsheet tableau of concept alternatives versus evaluation criteria shows which evaluation criteria are driving the decision, a fact often obscured by numeric schemes.

• The method makes it easier to evaluate a large number of alternatives using a variety of evaluation criteria.

All utility function methods, as well as other decision support algorithms, are essentially qualitative analysis done with a quantitative cast. Curiously, Pugh's method remains strictly qualitative right up to the last moment, when the pluses, minuses, and sames are added up to produce a score. This allows a qualitative backtrack through the data, even up to the final matrix to allow the team to see which criteria are really driving the decision.

After a particular software concept has been selected from a list of possibilities generated from TRIZ and brainstorming techniques, the stage is set to carry out risk assessment of the proposed development concept, using FMEA and other techniques (see Chapter 13), before proceeding to Phase 2, which deals with Robust Design (see Chapters 16 and 17).