Selecting and Testing Solutions

Overview

Purpose of these tools

Deciding which tool to use

Sources of solution ideas

  Tips 
  • Focus on one root cause at a time

  • Start with the root causes determined to have the biggest contribution to the problem statement

Benchmarking

Highlights

Sources of benchmarking data

Types of benchmarks

 

Pros

Cons

Internal/Company

Establishes a baseline for external benchmarking Identifies differences within the company Provides rapid and easy-to-adapt improvements

Opportunities for improvement are limited to the company' s internal best practices

Direct Competition

Prioritizes areas of improvement according to competition Initial area of interest to most companies Best used for in-depth studies as supplement to competitive intelligence studies

Often a limited pool of participants Opportunities for improvement are limited to "known" competitive practices Potential antitrust issues

Industry

Provides industry trend information Provides management with a conventional basis for quantitative and process-based comparison

Opportunities for improvement may be limited by industry paradigms

Best-in-Class

Examines multiple industries Provides the best opportunity for identifying radically innovative practices and processes Provides a brand new perspective Free exchange of information more likely to occur

Often difficult to identify best-in-class companies Sometimes difficult to get best-in-class companies to participate

Tips on solution selection

  1. Generate potential solutions: It is best to exhaust all ideas that the team can produce. Often, the best potential solutions emerge toward the end of intense creativity efforts, when everybody has many details fresh in mind related to the problem at hand.

    • The more potential solutions available to explore, the more opportunities for uncovering ideas for improvements

  2. Narrow the list and synthesize: If you have more ideas than you can reasonably act on, use affinity diagrams (p. 30), multivoting (p. 31), or other techniques to identify themes and trim the list. Whenever possible, synthesize by combining the best characteristics from alternative options to generate a single stronger solution. (Be sure to develop and use evaluation criteria, see below.)

  3. Select the best solution: Selection involves deciding which solution improvements to implement.

  Tip 
  • The synthesis and selection process is iterative. Work through it once to generate some ideas, evaluate those ideas, then brainstorm off the best options or newer ideas to see if you can develop even better solutions.

Developing and using evaluation criteria

Highlights

Example criteria

Weighting criteria

Not all criteria are created equal. When evaluating alternatives, it' s best to weight the criteria by assigning numerical values that indicate relative importance.

  1. Identify the criteria you want to use

  2. Assign a numeric value to each criteria that indicates its relative contribution towards achieving your goals

    • You can use pairwise comparisons (p. 261) to develop weights if you want

    • Make sure there is enough spread in the values to allow you to distinguish between truly good and truly poor ideas

      Ex: If "Providing on-time delivery" is a lot more important than "using existing software," give delivery a value of 10 and software a 3 or 4

      Ex: If on-time delivery is only slightly more important than using existing software, give delivery a 10 and software a 7 or 8

  Tip 

Questions to help determine the evaluation criteria:

  • What will the best solutions look like?

  • What are the barriers to implementation?

  • Which type of solutions will be the cheapest to implement?

  • Which type of solutions will be the most dramatic? The most visible?

  • Which type will show the fastest results and deliver the most "bang for the buck?"

  • Which type will meet the least resistance and be the easiest to put in place?

  • What factors are most likely to affect your department and other departments?

Solution selection matrix

Purpose

When to use a solution selection matrix

How to create and use a solution selection matrix

  1. Remove show stoppers from your list of alternative solutions. Solutions with components that would prohibit their implementation should be removed prior to performing additional analysis.

    • Ex: addresses a defect but results in a large adverse impact on customers

    • Ex: directly conflicts with the organization' s strategy

    • Ex: goes beyond the scope of the charter

  2. Consider organization fit for each remaining idea. The solution must be capable of obtaining management commitment, and fit with customer needs, strategic objectives, organizational values, and the organization culture. Eliminate any options that do poorly on questions such as:

    • Management commitment—can you develop support for this idea?

    • Strategic factors and organizational values—is this idea consistent with our one-and two-year goals? five-year goals?

    • Operating and management systems—does the potential solution complement or conflict with our decision making, accounting, communication, and reward systems?

  3. Determine project goal impact for each remaining idea. Each potential solution must be evaluated on its ability to reduce and eliminate the root causes of poor performance. The solution must have a sufficient impact on the process to achieve the desired performance levels. Solutions that can' t produce the desired results must either be altered to meet the goals or removed from consideration.

    • Standalone (independent) solution ideas: These are solutions that by themselves are capable of satisfying the project goals or, due to their uniqueness, can' t be combined with other solutions

    • Coupled solutions These are solutions that in isolation are not capable of satisfying the project goals, but are capable of being combined with other solutions

  4. Narrow the list. Alternative techniques include:

    • Multivoting—Use to trim a list down to a manageable number (4 to 7). See p. 31 for instructions.

    • CDAM—Evaluate the alternatives and see if you can Combine, Delete, Add, or Modify them.

    • Pairwise Ranking—Qualitatively rank solutions against each other. See p. 261 for instructions.

    • Force Field Analysis—Evaluate forces working for and against the various alternatives.

  5. Enter criteria and top solution alternatives into a solution selection matrix

     

    Process Impact

    Time

    Cost vs. Benefit

    Other

    Total Score

    Rank

    Weight

    2

    2

    3

    1

    		    

    Opt 1

    8

    8

    10.5

    4

    26.5

    2

    Opt 2

    14

    18

    22.5

    7

    32.5

    1

    Opt 3

    2

    4

    21.0

    1

    28

    3

  6. Score alternatives on each criteria

    • Process impact: Score the impact the solution will have on the problem (the ‘Y’), versus the other solutions.

    • Evaluate time impact: Decide whether it' s important to understand (a) The total amount of time needed to design and implement the solution, or (b) How long it will be before a process is actually performing at desired levels (weeks, months, etc.). Rank or score accordingly. An alternative is to express time impact in terms of the expected fulltime resources (FTEs) necessary to implement the solution.

    • Evaluate cost/benefit impact—What is the relationship between the total costs and the expected benefits the business will realize as a result of the implementation?

    • Evaluate other impacts— What are the impacts that the organization wants to keep visible during the decision-making process? Common examples include safety, business risk, and morale.

  7. Use FMEA (p. 270) or any risk-evaluation technique commonly used in your company, as appropriate

  Tips 
  • After a lot of discussion on each of the solutions, team members may find themselves gravitating toward one or two favorites. Remaining objective is important—a bias or uninformed preference may cause the team to overlook excellent cost-beneficial solutions.

  • Remember, nothing gets thrown away. These tools help the team focus on likely solutions—but later you may find it helpful to revisit ideas that were set aside previously in order to spark more creative thinking.

Pairwise ranking

Highlights

To use pairwise ranking to select solutions…

  1. Identify the solution ideas you want to compare

    • List them on a flip chart or white board so that everyone can see them

    • Label them with numbers or letters (Ex: A to E if you have five options)

  2. Draw a matrix with as many columns and rows as there are items (Ex: five rows and five columns if you have five items)

    • This matrix will be used to record the results, so it should be visible to everyone (on a flip chart, white board, overhead projector, etc.)

    • Label the columns and rows with the assigned numbers or letters

    • Color in or otherwise mark off the diagonal boxes that represent comparing an item to itself

  3. Review or develop criteria

    • During the comparison, you' ll be asking how one solution compares to another, which means you all need to agree on what criteria will be used

  4. Compare each item to every other item, one at a time, until you' ve filled out the upper or lower half the matrix (doing both sides would be repetitive)

    • For each comparison, ask which solution is better

    • Use a show of hands or other voting method to select which is better

  5. Record the results

    • Simply put the letter or number of the preferred option in the box

      Ex: In one comparison, Option A was voted as being better than option B

  6. After completing all comparisons, tally the number of times each option won out. Interpret the results.

    • In the example above, A = 3, B = 1, C = 0, D = 4, and E = 2

    • This would indicate an overall preference for item D.

    • Explore ways to improve option D by incorporating elements of the other strong options. Here, for example, ask why Options A and E are stronger than B or C. Can you work elements of A and E into the solution?

  7. OPTIONAL: Sometimes the interpretation is easier if you tally the number of votes each item gets instead of the number of times it "won." Here' s a portion of the above example, this time with the number of votes recorded:

    • This increases the spread between items, emphasizing strong differences and giving added weight to items that people feel strongly about

    • In this example, the tally might end up something like: A = 12, B = 5, C = 0, D = 18, E = 8

Using pairwise ranking to weight criteria

Uses the same method above except compares potential criteria for evaluating solutions, not the solutions themselves.

  1. Identify and label (letter or number) the criteria.

  2. Create a table and compare each criterion against each other criterion. In this case you' re asking "Which of these criteria is more important?"

  3. Tally the number of times each criterion "wins" then add one point (to make sure there are no 0s in the table).

  4. Tally the total number of points.

  5. Determine the % of points each criterion received.

  6. Assign weights (usually on a scale of 1 to 5 or 1 to 10, with higher numbers being better)

Cost evaluation

Most solutions will need to be evaluated based on the cost of implementation.

Impact effort matrix

Impacts to consider

Pugh matrix

Purpose

A decision-making tool to formally compare concepts (processes, services, products) based on customer needs and functional criteria.

When to use a Pugh matrix

How to create a Pugh matrix

  Note 

A Pugh matrix analysis is performed iteratively, with each time through called a "run." The first time through ("first run") you want to develop a small number of strong solutions. Then you repeat the full analysis to confirm the initial results.
  1. Develop potential solutions (alternative concepts). Capture each, as appropriate, in…

    • Drawings

    • Word descriptions

    • Other characterizations of potential solutions

  2. Identify criteria

    • See p. 256 for general instructions. Focus on those most relevant to the purpose of your current effort (customer requirements, design parameters, project goals, etc.)

  3. Weight the criteria (see p. 257)

  4. Select one alternative as the baseline

    • Often the current method or model, but could be any of the alternatives

  5. Prepare an evaluation matrix

    • List alternative solutions across top of matrix

    • List criteria in first column

    • Importance ratings in second column or in last column

    • Enter "B" in all cells under the Baseline alternative

  6. Score each of the alternative solutions against the baseline:

    + or ++ means better or significantly better than baseline

    − or −− means worse or significantly worse than baseline S means about the same

  7. Sum the pluses (+), negatives (−), and "sames" (S)

  8. Multiply the counts of the +' s,−'s, and S's by the Importance Rating (weights) and sum vertically to compute the Weighted Total for each concept

    • Do not treat the numbers as absolute

  9. Focus first on the alternative(s) with the most pluses and the fewest minuses

  10. Look for strengths and weaknesses and determine how to attack the weaknesses

    • What is needed to reverse the negatives?

    • Does the change reverse any of the positives?

    • Can strengths of other alternatives be incorporated to address or reverse the negatives?

    • If another alternative is "S" or "+", perhaps that alternative contains an idea (solution) that could improve the selected alternative

    • If a modified solution emerges, enter it into the matrix

    • Eliminate truly weak concepts from the matrix

  11. Examine positives

    • If you find a uniform strength exhibited across a number of solutions, either (a) the criteria are too ambiguous to discriminate between solutions or (b) some solutions are strongly related (are subsets) to the others

      • In the first case, decompose the criteria (divide them into subcomponents)

      • In the second case, decompose the solutions, then look for ways to recombine them

  12. Identify the strongest possible solutions and repeat analysis to confirm first-run results

    • If the first run is not confirmed, continue to analyze and improve the alternatives until a strong solution emerges

    • Conduct a confirmation run using the strongest solution as the baseline

    • If the second run confirms the first, proceed to Controlled Convergence

  13. Perform Controlled Convergence analysis: enhance positives and eliminate negatives to get a better solution than any of the original ideas

    • List product service and process attributes

    • List alternative solutions

    • Pick strong solution as baseline

    • Evaluate solutions against defined criteria

    • Evaluate solutions as better (+), worse (-), or same (S) compared to baseline

    • Assess the individual solution scores

    • Identify strong and weak solutions

    • Attack the negatives and enhance the positives

      • Attempt to reverse the negatives by improving designs

    • Enhance the positives of the weaker solutions

      • Attempt to improve the solution designs. Strengthen strong solutions with strong aspects of the weaker solutions

    • Abandon solutions that remain weak

      • Add new Solutions if warranted. Matrix should get smaller.

    • Rerun the matrix using the strongest Solution as a new baseline

    • Repeat the process until strong solutions persist through several runs

  14. Perform product and process design work if necessary

  Tip 

Pitfalls that often occur in the first run

  • Desire to cut-and-run prematurely terminates process

  • Doubts about the validity of process arise, especially as strong solutions emerge and stay strong

  • Disruptions by those who see the process as a competitive winlose contest, rather than a collaborative win-win effort

Other evaluation techniques

Simulation models—Ask for help from experts to develop computer simulations that model the current process and demonstrate how an improved process might operate.

Benchmarking—Identify the "Best-in-Class" organization in any process or capability you' re studying, regardless of what industry that company is in. For example, Memorial Hospital might look at Hilton Hotel' s guest registration process when evaluating its patient admission process. (See more on benchmarking on p. 254.)

Industry standards—Research organizations conduct comparative studies of industries and publish performance data. A project team at a refinery might reference research data on refinery capacities to assist their decision.

Internet research—Using the numerous search engines, teams can conduct competitive analysis or benchmarking with organizations worldwide.

Conducting a limited pilot—The team may elect to implement the solution through a limited pilot. Before and after measurements may be used to extrapolate the value if implemented on a larger scale. Consider the time to implement as well as the associated risks to the normal operation.

Controls assessment matrix

Project Objective: Accurate receipt of contracts via EDI

Risk: Information is lost in transaction

Sufficiency of Control: (x = excessive, a = adequate, i = inadequate)

Controls

Existing Y/N

Is it sufficient? If not, why?

Next Steps

a) Training

Y

a-Training conducted

What? Who? When?

b) Procedures not specific

N

a-Configured to prevent submission of transactions without required data

  

c) No Process for customer master

N

i-Customer master verification

  

Date:__ Prepared by:__ Reviewed by:__ Next Update:__

Highlights

To create a controls assessment matrix…

  1. List your highest priority risk

  2. List the controls that would help mitigate or eliminate the risk

  3. Determine if those controls exist

  4. Discuss the appropriateness and sufficiency of each control

  5. Identify actions needed to address any control deficiencies

  Tip 
  • Use the results as an input to the Process Control Plan

Failure Modes and Effects Analysis (FMEA)

Purpose

A structured approach to:

When to use FMEA

Types of FMEA

  1. Design FMEA: Analyzes a new process, product, or service design before rollout to understand how it could fail once released. Exposes problems that may result in safety hazards, malfunctions, shortened product life, or decreased satisfaction.

  2. Process FMEA: Used to improve existing transactional and operational processes to understand how people, materials, equipment, methods, and environment cause process problems. Exposes process problems that may result in safety hazards, defects in product or service production processes, or reduced process efficiency.

  3. System FMEA: Analyzes systems and subsystems in the early stages of concept and design.

    Process or Product Name:

    Prepared by:

    Responsible:

    FMEA Date (Orig) ____(Rev)

    Process Step/Input

    Potential Failure Mode

    Potential Failure Effects

    SEVERITY

    Potential Causes

    OCCURRENCE

    Current Controls

    DETECTION

    RPN

    What is the process step and Input under investigation?

    In what ways does the Key Input go wrong?

    What is the impact on the Key Output Variables (Customer Requirements)

    What causes the Key Input to go wrong?

    What are the existing controls & procedures (inspection and test) that prevent the cause of the Failure Mode?
                   

    0
                   

    0

How to perform FMEA

  1. Review the product, service or process

    • If working on a process, start with the steps that contribute the most value

  2. Brainstorm then sort possible failure modes

    • A failure mode is the way in which the component, subassembly, product, input, or process could fail to perform its intended function. They may be the result of upstream operations or may cause downstream operations to fail.

  3. List one or more potential effects for each failure mode

    • Answer the question: If the failure occurs, what are the consequences?

  4. Assign ratings for severity and occurrence

    • Severity of failure: 1-10, with 10 representing most severe impact on customers

    • Likeliness a failure will occur: 1-10, with 10 representing most likely to occur

  5. List current monitoring and controls for each failure then assign a detection rating to each

    • Detectability of failure: 1-10, with 10 representing least likely to be noticed given your current control methods

  6. Calculate a risk priority number (RPN) for each effect by multiplying the three numbers (severity * occurrence * detection)

  7. Use the RPNs to select high-priority failure modes

    • Prioritize actions so the highest RPNs are attended to first

    • Exception: Any failure with a severity rating of 10 must be worked on immediately because of impact on customers, even if it does not score a high RPN overall

  8. Plan to reduce or eliminate the risk associated with high-priority failure modes

    • Identify potential causes of the selected failure modes

    • Develop recommended actions, assign responsible persons

    • Look for both:

      • Preventive action: Steps that reduce the likelihood of a problem occurring at all, focused on reducing/eliminating root causes prior to occurrence

      • Contingent action: Measures implemented to limit the damage caused by a potential problem should it occur; focused on achieving the goal in spite of difficulties

  9. Carry out the plans. Document actions taken

  10. Recompute RPN

    Process or Product Name:

    Prepared by Page__of:__

    Process/Product FMEA Form

    Responsible:

    FMEA Date (Orig) ____(Rev)

    Process Step/Input

    Potential Failure Mode

    Potential Failure Effects

    SEVERITY

    Potential Causes

    OCCURRENCE

    Current Controls

    DETECTION

    RPN

    Actions Recommended

    Resp.

    Actions Taken

    SEVERITY

    OCCURRENCE

    DETECTION

    RPN

    What is the process step and Input under investigation?

    In what ways does the Key Input go wrong?

    What is the impact on the Key Output Variables (Customer Requirements)

    What causes the Key Input to go wrong?

    What are the existing controls & procedures (inspection and test) that prevent either the cause of the Failure Mode?

    What are the actions for reducing the occurrence of the cause, or improving detection?

    		  

    What are the completed actions taken with the recalculated RPN?

    Fill carafe with water

    Wrong amount of water

    Coffee too strong or too weak

    8

    Faded level marks on carafe

    4

    Visual inspection

    4

    128

    Replace Carafe

    Mel

    Carafe replaced

    8

    1

    3

    24
         

    8

    Water spilled from carafe

    5

    None

    9

    360

    Train employees

    Flo

    Employees trained

    8

    2

    7

    112
     

    Water too warm

    Coffee too strong

    8

    Faucet not allowed to run and cool

    8

    Finger

    4

    256

    Train employees

    Flo

    Employees trained

    8

    2

    6

    96
         

    8

    Employee not aware of new need for cool water

    7

    None

    10

    560

    Train employees

    Flo

    Employees trained

    8

    1

    8

    64
     

    Carafe not clean

    Foreign objects in coffee

    10

    Carafe not washed

    4

    Visual inspection

    4

    160

    Appoint inspector before storage

    Alice

    Vera is the new inspector

    10

    1

    4

    40
       

    Bad taste

    10

    Carafe stored improperly

    7

    training

    5

    350

    Create storage bin & train employees

    Alice

    New storage bin & employees trained

    10

    2

    3

    60

  Tips on scoring systems 
  • There are a wide variety of quantitative and qualitative scoring "anchors" that can form the basis for your rating scales. If you have difficulty, ask data experts to help you find examples you can use as models.

  • Two rating scales typically used:

    • 1 to 5 scale makes it easier for the teams to decide on scores

    • 1 to 10 scale allows for better precision in estimates and a wider variation in scores; it is more commonly used

Pilot testing

Purpose

To identify practical problems and failures in a chosen solution so those can be addressed before full-scale implementation

Key characteristics of a pilot test

A pilot is a test of a selected solution. This type of test has the following properties:

The pilot should test if the process meets both the design specifications and customer expectations.

How to pilot a solution

Phase 1: Plan

Phase 2: Review design

Before conducting the pilot, review your plans to…

Phase 3: Finalize design and implement

Phase 4: Evaluate the test and verify results

  Tips   
Related

Категории