Design for Trustworthy Software: Tools, Techniques, and Methodology of Developing Robust Software

  • Poka yoke, which means mistake-proofing or fail-safing, recognizes that human errors are unavoidable but do not necessarily have to result in defects.

  • Pioneered by Shigeo Shingo, poka yoke employs a set of proactive measures, particularly a 100% inspection at the source to detect process errors before they cause defects. It emphasizes prevention and includes stopping the process upon detection of process errors that may cause defects downstream. The process is restarted only after the cause has been identified and eliminated.

  • Complexity and mistakes are strongly linked. As such, it is best to eliminate the causes of potential mistakes substantially by complexity reduction before a poka yoke deployment.

  • The five principles of effective poka yoke deployment are 100% inspection at the source, appropriate upstream control measures, rapid implementation, people focus and teamwork, and problem-solving approach.

  • Organizations that rely merely on variation-focused initiatives such as Six Sigma will fail to achieve a Six Sigma quality level unless they also find effective ways to address mistakes and complexities (the "mother of all defects"). Trustworthy software must be robust vis-à-vis all three causes of defects: variations, mistakes, and complexities.

  • Poka yoke should not be approached as a panacea for all kinds of mistakes. It is also important to understand that poka yoke may be more suitable in certain situations than others.

  • Mistakes can be defined as executing a prohibited action, failing to perform a required action, or misrepresenting information essential for the correct execution of an action.

  • Rare mistakes do occur, even in high-capability processes. The fact that mistakes are rare in many processes means that traditional sampling and statistical methods are not useful in estimating their frequency.

  • As many as 80% of the nonconformities in complex systems can be attributed to mistakes.

  • Mistakes can be effectively described only in terms of probability: Mistakes either occur or they do not. Because every distribution describing variation can be converted to probabilities, the only universal method of describing both variation and mistakes is probability.

  • Trustworthy software boils down to freedom from defects (nonconformance). As the process capability improves, mistakes play an increasingly crucial role in determining product quality. Furthermore, the collective impact of these mistakes, commonly prevalent in complex systems, can be potentially catastrophic.

  • Avoiding mistakes rather than SQC-based variation control has been a key instrument of conformance quality in nuclear power plants and air traffic control systems, where consequences of failure could be catastrophic.

  • A key strategy in mistake-proofing is to control complexity.

  • Software is intrinsically more complex than hardware because it has more states, or modes of behavior. An integrated enterprise business application system, for example, is likely to have 2,500 or more input forms. No machine has such a large number of operating modes. Computers, controlled by software, have more states (that is, larger performance envelopes) than do other, essentially mechanical, systems. Thus, they are intrinsically more complex.

  • We do not understand the nature of complexities in software and are not likely to understand or measure complexities anytime soon. A number of approaches have been taken to calculate or at least estimate the degree of complexity, but the simplest is LOC.

  • Even though we may never fully comprehend complexity and measure it, a number of identifiable surrogates exist: number of inputs, number of outputs, number of states, LOC, module length, number of design changes, complexity level, number of features, defect rates, number of components, lack of discipline on the part of software developers, and the duration of the development process.

  • Hinckley reports a linkage between assembly time and defect rate and proposes time as a measure of complexity. Care must be taken when using time as a measure of complexity, because worker skill, training, and the workplace strongly influence how long it takes to perform similar tasks. Generally, task complexity can be reduced by product designs that take less time to complete.

  • The complexity reduction framework consists of the Robust Development Model, value analysis, standardization, documentation, the 5S system, decomposition, and smarter domain-specific languages and formally based tools.

  • There are three types of mistakes: ones that have occurred and that resulted in defects, ones that have occurred but have not yet resulted in defects, and ones that have not yet occurred but may occur if corrective measures are not taken. The payoff increases as we move from prevalence of the first type of mistake to the third type.

  • These three mistakes require inspection to identify, but the implications are totally different. Shingo categorizes inspections as judgment inspections, informative inspections (SQC, SuCS, SeCS), or source inspections. The payoff in source inspection is the highest; only this can make zero defects possible. Both SuCS and SeCS are only informative inspections and cannot prevent the occurrence of even one defect. It is therefore desirable to deploy 100% inspection at the source with poka yoke. The use of SuCS and SeCS should be limited to instances constrained by technical or financial limitations.

  • Almost all mistakes are caused by human errors: forgetfulness, slowness, misunderstanding, lack of standards, errors in identification, errors made by amateurs, intentional errors, inadvertent errors, and surprise errors.

  • Hinckley has proposed a six-step methodology, quality by control of mistakes (QCM), for an efficient approach to identify a poka yoke measure for problem-solving.

Категории