Software Estimation: Demystifying the Black Art (Best Practices (Microsoft))

13.1 Group Reviews

A simple technique for improving the accuracy of estimates created by individuals is to have a group review the estimates. When I have groups review estimates, I require three simple rules:

• Have each team member estimate pieces of the project individually, and then meet to compare your estimates Discuss differences in the estimates enough to understand the sources of the differences. Work until you reach consensus on high and low ends of estimation ranges.

• Don't just average your estimates and accept that You can compute the average, but you need to discuss the differences among individual results. Do not just take the calculated average automatically.

• Arrive at a consensus estimate that the whole group accepts If you reach an impasse, you can't vote. You must discuss differences and obtain buy-in from all group members.

The improvement in accuracy from this simple technique is significant. Figure 13-1 illustrates the results across 24 groups of estimators I've worked with.

Figure 13-1: A simple review of individually created estimates significantly improves the accuracy of the estimates.

The individual estimates in Figure 13-1 average a Magnitude of Relative Error of 55%. The group-reviewed estimates average an error of only 30%. In this set of estimates, 92% of the group estimates were more accurate than the individual estimates and, on average, the reviews cut the error magnitude approximately in half.

How many experts are enough? Studies in other fields have found that the use of 3 to 5 experts with different backgrounds seems to be sufficient (Libby and Blashfield 1978, Jørgensen 2002).

In addition, it's useful to find experts with different backgrounds, different roles, or who use different techniques (Armstrong 2001, Jørgensen 2002).