The Art and Business of Speech Recognition: Creating the Noble Voice

The results of a usability test are only meaningful if the test subjects are representative of the people who will actually use the application. This may seem obvious, but I've often seen companies use their own employees as test subjects. Sometimes they can give good feedback, particularly if they are the employees who regularly work closely with customers, such as bank tellers or customer service representatives, or when the company is testing an application to be used by the company's employees.

However, more often than not, employees want the system to work in a way that works best for them ”not for the actual users! For example, if an airline application said, "Please say your reservation code," an airline employee might say, "Well, it should say 'PNR' ”for Passenger Name Record. That's what it's called." However, most people who fly don't know what a "PNR" is (though they do understand the concept of a "reservation code"). Employees may be the lowest -cost and most easily available test subjects, but they're clearly not the optimal choice, unless of course the application is intended for employees.

So what kind of people do we want to test an application? We want a group of subjects who approximate the intended user population ”the closer, the better. For example, let's say we are testing a trading application designed for brokers in the southern U.S. and our test subjects are consumers from northern California who sometimes make trades. The results would not be reliable.

Instead, our test subjects not only should be stockbrokers from the southern U.S., but also should match the gender, age profile, net worth, and brokerage experience of our target population. So if 20% of brokers in the Deep South are under age 30, 60% are under 40, and the rest are under 50, then we'd want to make sure that our test subjects match that age breakdown. The same holds true for the male/ female ratio. It makes no sense, for example, to have men testing an application that will be used almost exclusively by women.