Design for Trustworthy Software: Tools, Techniques, and Methodology of Developing Robust Software

2017-07-07 02:10:07

This book advocates moving the quality-related aspects of development as far upstream in the development process as possible. QFD provides tools and techniques to help with even the "fuzzy front end" of the development process. The Robust Software Development Model (RSDM) presented in this book provides a powerful framework for developing trustworthy software in a time- and cost-effective manner. A key aspect of Specification or Functional Design, performed by systems analysts in concert with the potential end users of the software, is to determine who the targeted customers or intended users are for the software solution we are developing, why the customers or users need a solution (what user problems the software must solve), and therefore what the software should do and how well it should do it.

So, what role can QFD play in an RSDM? QFD's contributions to the Seven Components of RSDM are as follows:

1.	QFD can make interactions with customers more efficient and productive, and it offers tools and techniques for identifying spoken and unspoken requirements throughout the software development process.
2.	QFD can make feedback and communication more effective between developers and across development phases.
3.	QFD enables the optimization of design for reliability (or other software quality attributes) from a customer perspective, and it has methods for reducing the time (and therefore the cost) of development.
4.	QFD supports the identification of opportunities for "quick hits" with incremental development methods, and better "aiming" of such methods.
5.	QFD can be performed in a step-wise fashion and can be tailored as needed by an organization or a project team.
6.	QFD offers tools and techniques for risk analysis from a customer perspective throughout the development process.
7.	QFD enables object-oriented development to be customer focused and value driven.

In Chapter 2's discussion of the Design for Trustworthy Software (DFTS) process, software quality is defined as the degree to which a system, component, or process meets customer or user needs or expectations. Although all five major components of trustworthy software are areas where QFD can play a role, the most significant area is customer responsiveness. If the software is of no value to the customer, the reliability, safety, security, or maintainability of the software is irrelevant to the customer. Customers are only interested in the trustworthiness of software that is worth something to them. To understand customers' definitions of value, we have to "listen to the voice of the customer"a concept that QFD created and introduced.

To develop trustworthy software and to gain customers' trust, we must first meet customers' needs, whether stated or unstated. We must be able to identify these needs correctly on an iterative basis throughout all phases of the software development process.

But how can we meet unstated needs? And how can we ensure that our software, when delivered, will actually be trustworthy in the eyes of its users? QFD was created to address exactly such questions.

What's Different about QFD as a Quality System?

The traditional role of methods such as Statistical Process Control and the DMAIC problem-solving approach of Six Sigma has been to improve the quality of a product or process after it has been produced or established. Methods such as Statistical Process Control (SPC) and the DMAIC problem-solving approach of Six Sigma are excellent in such situations. They can take an existing situation and efficiently reduce "negative quality" such as defects, problems, and variability. In the late 1960s in Japan, Dr. Yoji Akao saw that such methods were not enough, however. Two fundamental insights led to the development of QFD:

It is important to design in quality; that is, to produce quality from the beginning, and not to design or produce something and then correct it.

It is necessary to consider quality from the perspective of the customernot just from the technical perspective of the engineer or developer.

These insights ultimately led to the development of QFD as a modern quality system, focused on understanding what value means to the customers or users we intend to satisfy.^[1] Therefore, QFD uses tools and techniques that take customer statements and actions as input, in addition to the traditional methods based on measurements of what's wrong with what already exists (see Figure 11.1).

Figure 11.1. Traditional versus modern quality systems.

Traditional quality-improvement and problem-solving methods focus on reducing defects. Dr. Deming described such approaches as "scraping burnt toast." And even with all defects removed (zero defects), are we assured that our customers will be satisfied? No. Satisfaction is not merely the absence of defects. Rather, satisfaction requires the presence of value, as determined by the customer. Modern quality methods, such as QFD, focus on adding value to the customer, which requires new tools and techniques.

Traditional quality-improvement and problem-solving methods use data (measures from an existing product or process) to find the root cause of defects or problems and remove them or solve them. In order to use these approaches, you must first develop somethingthen you can test it and improve it. In contrast, QFD's aim is to understand the customer's needs and use that understanding to drive design and developmentto ensure customer satisfaction on the first pass through development.^[2] So, you can use QFD for new-product development, even for the initial development of products and software that are new to a customer, a company, or the world.^[3]

Further, in contrast to the assumptions of many approaches to software requirements, QFD does not assume that customers understand their needs, or that customers can tell us all of their needs. Note that many iterative or evolutionary development approaches begin with an assumption that it is not possible or practical to get a sufficient set of requirements from customers such that developing to those requirements would satisfy customers. Thus, iteration is used as a strategy to obtain feedback from customers on an ongoing basis in order to redirect the development effort in mid-flight. This series of small build-and-correct cycles allows developers to proceed even when only a weak understanding of what is needed is available (from customers or analysts). The application of QFD tools and techniques to requirements discovery creates a much better understanding of requirements, much more quickly. Therefore, the need for and the number of iterations required during development is drastically reduced (see Figure 11.2).^[4]

Figure 11.2. Iterative methods alone, and with QFD. Fewer iterations require less effort and take less time. QFD tools, applied to better discovery of customer needs at the front end of the development process, can reduce the number of iterations required, and thus development time and effort.

The History of QFD

QFD originated more than 20 years ago in Japan for the efficient development of products and services that satisfy customers. Dr. Yoji Akao and other leading quality experts in Japan developed the QFD tools and techniques and organized them into a comprehensive quality system.^[5]

Organizations throughout North America have used QFD since 1984, with cross-functional teams and concurrent/simultaneous engineering, and on services, products, and the product development process.^[6], ^[7], ^[8] In certain industries, such as the automobile industry, QFD use is now practically universal. And as companies such as Toyota and Ford demonstrated the effectiveness of QFD, its use in the software industry became inevitable (see Figure 11.3).

Figure 11.3. The first 20 years of QFD in Japan and North America. Note that QFD development preceded its use at Toyota in the 1970s and the first use of a matrix came before Mitsubishi Heavy Industry's application at its Kobe shipyards. The first QFD book written in English was Bob King's Better Design in Half the Time, published in 1987.

The History of Software QFD

Why apply QFD to software? In almost all cases, software is developed to produce a result for the organization(s) that funded the software's development. And in most of those cases, software is built to satisfy customers, both internal and external. To satisfy customers better (and/or faster), we must deliver value to them more efficiently. To do this, we need a framework for value, and a process to ensure its efficient delivery. QFD was developed over the past 40 years for this purpose^[9] and it has been applied to software for almost two decades.^[10]

In Japan

Since 1982, Japanese software organizations have been using QFD with impressive results.^[11] Dr. Tadashi Yoshizawa and other leading software quality experts in Japan pioneered Software QFD.^[12] NEC IC Microcomputer Systems Ltd., which received a Deming Prize in 1987 for its world-class software-quality efforts, is a sterling example of the power of Software QFD.^[13] In 1989, with support from the Ministry for International Trade and Industry (MITI), the Information Processing Promotion Industry Association released a two-volume, 360-page book on Software QFD to further the development of high-quality software by Japanese firms.^[14] These impressive efforts to combine sophisticated software engineering^[15] with state-of-the-art quality systems, such as QFD, are continuing.^[16]

In North America

Since 1988, North American software organizations have been using QFD.^[17] Firms such as AT&T Bell Laboratories,^[18] DEC^[19], ^[20] Hewlett-Packard^[21], ^[22] IBM,^[23] Texas Instruments,^[24] and others have reported good results with Software QFD. QFD has become an essential part of TQM for world-class software organizations^[25] and today it is part of DFSS as well.

So, What Is QFD and Why Do We Need It?

To understand what QFD is and why we need it, it helps to consider a very simple model of the development process.

Analysis phase. What the customer wants, documented in a specification.

Design phase. How the product will work, documented in a design.

Development phase. Make the product and test it.

Implementation. Deliver/install the product and begin supporting it.

Now consider two organizations with the same resources and the same time pressures to develop a product to satisfy a customer.

Incoherent Development

The first organization proceeds as usual (see Figure 11.4a). It gathers requirements, documents them in a specification, and attempts to meet all the requirements throughout the development process. But like most software development organizations, this organization does not have enough resources or time to meet all customer requirements and provide best efforts throughout development. Even if the spec notes the high importance of some of the customer's requirements, there is no method to communicate such information to all the developers and through all the phases. Nevertheless, some items in each phase receive special attention, but this decision is made independently by those involved in each step of development; furthermore, by chance, some items that received best efforts in one phase also receive best efforts in a later phase. The result is a product that does not fully represent the effort the team put in, as most of the team's best efforts are wasted by lesser efforts upstream or downstream. Also, the product is strong in terms of meeting the requirements of greatest importance to the customer, but only by luck. Because the results depend on chance, a large organization with many development teams will have a few successes due to good luck, and a few clear failures due to bad luck. In both cases, the efforts of the team are not responsible for the result.

Figure 11.4A. An incoherent development process. The boxes on the left represent customer needs. Within each phase, best efforts are applied based on local judgments (and without regard to best efforts in the preceding phase). Over the life of the project, some best efforts align by chance, producing a product with strengths that are not predictable and that align with customer importance only by luck. Without a method to communicate what is most important to the customer end to end, alignment of best efforts to customer priority, and across phases, will be random.

Coherent Development

The second organization proceeds with QFD (Figure 11.4b). It uses QFD to discover the customer's voice and to gather the customer's requirements, including the priority the customer places on those requirements. There are not enough resources or time for all customer requirements to receive best efforts throughout development, so the team concentrates on the small number of high-value requirements. It uses specific formal and informal methods to communicate the high-value needs and their priority to all developers and through all phases. The high-value items receive best efforts in every phase, consistently.

Figure 11.4B. A coherent development process. QFD is used to discover truer customer needs including their importance. This is communicated end to end throughout the development process by both formal and informal means, and everyone concentrates their best efforts where it matters most to the customer. The result is a product with predictable strengths on items of greatest importance to the customer: a high-value solution, produced efficiently.

The result is a product that shows the team's true capability. They performed at their absolute best, end to end, on the items that mattered most to the customer. For the same amount of resources and time as the first organization, the second organization has produced a result that is clearly superior. And because they used a systematic process to produce this result, they can repeat their success, as can all other development teams in the organization.

A Focus on Priority

Why is focus so important to QFD? In order to satisfy customers when our time and resources are limited, we must be efficient. Consider a customer with many requirements, as shown in Figure 11.5.

Figure 11.5. Accurate measurement of requirements priority reveals a Pareto distribution.

So to improve our development process for greater customer satisfaction, we need to apply our best efforts on the small number of high-value requirements. Much of the effort spent on doing a great job on the many low-value requirements has no impact on customer satisfaction.

Some customer needs are more important than others. When the importance of needs is accurately measured, such as with the Analytic Hierarchy Process (see Chapter 8), it is clear from the priorities that a vital few requirements are as much as an order of magnitude more important than the many requirements of almost trivial importance. Performance on the trivial requirements doesn't matter. Only performance on the important requirements has a significant effect on customer satisfaction. With limited resources or time, it is essential to do your best on what matters most to customers. On the many trivial requirements, do what you can, time and resources permitting.

Most analysis approaches concentrate on requirement completeness. The assumption is that we should discover, document, and deliver all the customer requirements, and that this will satisfy the customer. In contrast, QFD focuses on requirement sufficiency. The assumption in this case is that if we concentrate our available efforts on the most important requirements, we have our best chance at satisfying the customer. The aim is to deliver a sufficient level of performance on a sufficient number of high-value requirements to satisfy the intended customer.

In Software QFD, software requirements of low value, but requiring high effort to develop, will be set aside to free up effort for the high-value requirements.^[26] They are intentionally not implemented. If low-value requirements are missing, customers may notice, they may complain, they may even insist they be added in the next upgrade, but they will still buy, accept, and use the software. The customer's interest, choice, and loyalty are won or lost on how the software performs on the small number of high-value requirements. Most requirements will have no significant impact on customer satisfaction because they just aren't that important to the customerthey're of low value.

QFD Defined

So, what is QFD? Japan Industrial Standard Q 9025 defines QFD as follows:

... [a] supporting technique for organizations to achieve effective and efficient performance improvement of management system and provides methods for deployment and realization of the voice of the consumer from product quality characteristics and product elements to process elements. Furthermore, it provides methods for identifying operation or job function that is important in assuring product quality. This standard has been designed for use by organization requiring the following.

Items from the perspective of new product development are the following:

Identification of voices of the consumer toward new product to be developed and compilation of such voices in the form that can be viewed easily

Identification of the design elements necessary to realize markets needs

Quantitative assessment of required quality and quality characteristics to be prioritized in product to be developed

Determination of design quality of the product in the planning and development stage

Early identification of technology to become bottleneck in development and review into solution of the problem

A grasp of management of the new product development process from a total perspective including quality, cost, productivity, and reliability^[27]

QFD is a modern quality system composed of deployments (subsystems), each focused on one dimension of the development process (such as reliability). These deployments in turn are composed of a series of tools, organized in a sequence to address the questions and concerns of that dimension throughout the project and the development process.^[28]

QFD Deployments

In this chapter, the term QFD refers to the entire quality system shown in Figure 11.6, not just to the first box, which represents the "House of Quality" matrix.

Figure 11.6. Deployments of QFD. QFD is a quality system with many subsystems. Task deployment is the application of QFD to the development process itself. It is used to tailor QFD to a specific organization. Comprehensive quality deployment refers to the QFD activities applied during a project. Specific deployments address the dimensions of development that are important for that project, such as quality, reliability, safety, security, and maintainability in a DFTS organization.

The Four-Phase Model of QFD

The four-phase model of QFD was popularized by a Harvard Business Review article, "The House of Quality."^[29] Presenting an early understanding of QFD, the model was widely adopted by automotive suppliers, for whom it had been developed. Parts suppliers at the time were operating in a build-to-print environment. Their auto company customers supplied them with a detailed design and asked them to build the corresponding parts.^[30] Figure 11.7 shows the four-phase model.

Figure 11.7. The four-phase model begins with the "House of Quality" and deploys quality characteristics into parts, the specific equipment to make those parts, and the settings and procedures for those parts-making machines. As this is a tailored QFD process for parts suppliers in a build-to-print environment, no design activities are supported, making this an inappropriate model for software development.

When the design is a given and cannot be changed, the only decision a parts supplier needs to make is how to build the part betterhow to best upgrade an existing model of the product. The four phases in this approach are as follows:

Phase 1: The "House of Quality" customer attributes/quality characteristics matrix. This matrix answers the question, what does a "good product" mean to our customers? The definitions of goodness contained in the customer attributes are mapped into the quality characteristics of the product. So this is where the "voice of the customer" represented by the product attributes is translated into the language of the product quality characteristics.

Phase 2: The Parts Deployment quality characteristics/parts characteristics matrix. This matrix answers the question, what parts of the product deliver the quality characteristics our customers want? Critical quality characteristics are mapped into parts and their characteristics. (Note that the design, and therefore the necessary parts, is already determined. There is no design in the software development sense in this model, as a parts supplier in a build-to-print environment cannot do systems design; it can only upgrade the existing product. It is constrained to only make changes within parts or within its production process.)

Phase 3: The Process Planning part characteristics/process parameters matrix. This matrix answers the question, where in our manufacturing process can we affect the critical parts characteristics? Critical parts characteristics are mapped into process steps and parameters. So this is where the "voice of the customer," translated into critical process steps and parameters, reaches the factory floor.

Phase 4: The Production Planning process parameters/production requirements matrix. This matrix answers the question, what should the production plans, procedures, and inputs be for the key process operations to produce the key parts (with their critical characteristics) to satisfy the customer? So now the "voice of the customer" has reached the machine operators, and it determines the settings on the production machinery.

At each phase, only the most important items are deployed to the next phase. (Otherwise, the matrices can become unmanageably large.) In this way, QFD is helping the entire production process end to end to focus best efforts on what matters most to the customers. So what is actually conveyed by the "voice of the customer" is not the content of what the customer saidthat changes from customer attributes to engineering characteristics, and so on, to machine settingsbut the importance to the customer, the priority. So across the dimensions of development, represented by the rows and columns of the matrices in the model, the knowledge of what's most important to the customer is communicated.

The "House of Quality" Matrix

When they think of QFD, many people think of the "House of Quality" matrix. So named because it resembles a house (if you use your imagination), this matrix is intended to be the bridge between the world of the customer and the world of the developer. It is here that traditionally the "voice of the customer," or customer needs, are translated into the corresponding quality characteristics and capabilities that the solution will require (see Figure 11.8).

Figure 11.8. Components, or "rooms," of the traditional "House of Quality" matrix. When first introduced to North America in the mid-1980s this was how the matrix was described for hardware and model upgrade applications. Today modern QFD uses a more sophisticated approach that leads to smaller and more focused matrices with greater accuracy that take less time and effort to produce.

The "House of Quality" matrix is an analysis matrix, and it is created before any design activities are performed. The matrix has several components, often referred to as rooms (listed here in order of completion):

Room 1. Traditionally, this room represents customer requirements (the whats). It is more accurate to say that this is the hierarchy of customer needs (the whys) with quantification (magnitude and priority).

Room 2. Traditionally, this optional room of customer competitive assessments is a table with fixed ordinal or interval scale weights. It is much more accurate (and valid) to use the Analytic Hierarchy Process (AHP) from Chapter 8 throughout this room. If there are no customer competitive concerns, such as for an internal project, this component is unnecessary.

Room 3. Traditionally, this room represents quality characteristics (the hows). It is more accurate to say that this is the hierarchy of the characteristics or capabilities that the solution must have to satisfy the customer (the whats).

Room 4. This room is the heart of the matrix. It maps the customer needs to the quality characteristics and capabilities, clearly showing the contributions the quality characteristics and capabilities make to customer needs. In Japan, Quantification Method III is used to reorder the rows and columns of this matrix, to enhance the patterns represented with symbols. Certain patterns of symbols are significant. Traditionally, the contribution strength was represented by a fixed 1-2-3 or 1-2-4 or 1-3-5 or 1-3-9 ordinal or interval scale. It is much more accurate (and valid) to use the AHP from Chapter 8 throughout this room.

Room 5. Traditionally, this optional room of technical competitive assessments is a table using fixed ordinal or interval scale weights. It is much more accurate (and valid) to use the AHP from Chapter 8 throughout this room. If there are no technical competitive concerns, such as for an internal project, this component is unnecessary.

Room 6. Traditionally, this optional "roof" of the "House of Quality" matrix is completed much later in the development process, by designers considering technical and physical conflicts among quality characteristics and capabilities for a given design. It is important to note that a "conflict" applies only to a particular design, and you can dissolve it using methods such as Teoriya Resheniya Izobreatatelskikh Zadatch (TRIZ; see Chapter 12 for an introduction to this powerful method for systematic innovation). If there are no technical or physical conflicts, such as for a software project, this component is unnecessary. (Most published case studies with roofs are poor examples of this very specialized component, and are good arguments for the use of TRIZ and other innovation methods to eliminate the conflict.)

Software QFD does not use the "roof" of the "House of Quality" matrix. The columns in the matrix are software or technical requirementsthe characteristics and capabilities which any solution must have in order to satisfy the customer, and therefore to be a "quality" solution. And these should be implementation free, or devoid of any assumptions of design or implementing technology. (Such design decisions occur later in the development process and are supported by subsequent QFD matrices and tables.) Therefore, the items in the columns cannot conflict, unless we have already created the design or are constrained in what design we can use (as may be the case for an upgrade of an existing product). In order for a conflict to exist, a design or implementing technology must be assumed. For example, in the classic hardware example of a car door, a conflict is presented between "ease of opening" and "sealing against rain". But this conflict exists only with a conventional door design. For tilt-forward doors, as on a Lamborghini Countach or Toyota Sera, no conflict exists. For gull-wing doors, such as on a Bricklin or DeLorean, no conflict exists. Also note that such conflicts are fundamentally physical conflicts, and there are no physical conflicts in software.

In software development, when we are making design decisions during the design phase and are choosing which implementing technology to use, it is useful and appropriate to examine logical or design conflictsand the roof can be helpful. But that is in the design phase, not in the analysis phase, and the "House of Quality" is an analysis phase matrix.

The "House of Quality" QFD

You are not performing QFD if all you are doing is creating a "House of Quality" matrix. This is the forceful pronouncement of Dr. Yoji Akao, the developer of QFD and still the foremost authority on QFD:

When a baby takes their first step, they are "learning to walk." But clearly they are not "walking" in the usual sense we use the word. A child of four can walk, run, skip, and jump. They are mature at walking. If they are still taking one step and falling down, it is cause for serious concern. Doing just a House of Quality for a first QFD project is unexceptional. Still just doing a House of Quality years later is unfortunate.

A "House of Quality" matrix is just one tool, only one step, and it could be the basis for QFD if you keep going, but just doing one matrix (of any kind) is not QFD. It is just creating a matrix. And a matrix is just one of the seven tools in the 7 MP toolbox of QFD (see Chapter 7 for information on all of the basic QFD tools).^[31]

"Kindergarten QFD"

Some software organizations have succeeded with Software QFD, only to stagnate and become a case of arrested development. These organizations learned about the "House of Quality" matrix of QFD and applied it successfully to their software projects. Then they expanded the number of projects using QFD, but not the amount of QFD that they were using. Some software organizations are still only using the "House of Quality" matrix years after learning about QFD. They crawl, but they have never learned to walk. They don't even know what walking is.

QFD Maturity

Once a team or an organization learns a bit about QFD, we can look at several factors to assess their success. We can look at the breadth or the extent of their use of QFD throughout the organization. We can examine the depth, or the degree to which they integrate and use QFD end to end throughout the development life cycle. We can see the height or power of their QFD process, by checking the number of deployments (subsystems) they use and the number of tools they employ in each. (This simple metric has limitations, but it works well for immature QFD organizations.) Combined, these measures tell us the team or organization's QFD maturity.

In competitive situations, a company using just one tool (such as the "House of Quality" matrix) will have a difficult time competing with an organization using several tools in an integrated way. And a company using just one deployment will wither in the face of organizations using multiple deployments. A deployment is a subsystem of the QFD system, with a series of tools sequenced to address one dimension of the development process, throughout the development process. For example, technology, or the means of implementing functional requirements, is one common deployment. Generally this deployment involves several tools applied at the architecture, application, subsystem, and module levels of detail. As a point of comparison, comprehensive QFD approaches contain the five basic deployments shown in Figure 11.6, plus additional special deployments.

One cause for concern is that in many U.S. software organizations, QFD development has halted at a low level; often at just the "House of Quality" stage. Is QFD so difficult to implement that only excellent organizations can succeed with it?

Consider the tools and techniques of structured analysis (SA), which were developed at about the same time as QFD. Although they were a tremendous step forward for software development (providing a structured approach to requirements specification), they were unwieldy, were labor intensive, and had numerous rough edges. (Many people just used data-flow diagrams.) Only good and determined organizations succeeded in mastering SA at that time. During the next decade, significant refinement and development occurred.

These changes did not invalidate the initial concepts of SA, but they did give practitioners greater power and assurance for less effort. This "next-generation" SA reduced the risks and made SA accessible and successful for a much wider audience.

QFD Evolution

QFD has evolved since it emerged in Japan, and moved to North America and then to the rest of the world.^[32] In the twenty-first century, QFD has been applied to QFD itself, to produce a "next-generation" Modern QFD. The basis for Modern QFD is the Blitz QFD approacha QFD method designed to be a better, faster, and cheaper QFD process than the traditional QFD approach.

Let's first look at some of the problems software organizations have experienced with traditional QFD in practice, and the causes of these problems. Then we'll consider how Modern QFD, based on Blitz, can prevent or minimize them, and why it is better, faster, and cheaper.