The Art and Business of Speech Recognition: Creating the Noble Voice

The transition from partial to full deployment can and should be a seamless process. When the system is stable and working well for callers , we can keep adding to the call volume until it's taking 100% of the calls 24 hours a day, 7 days a week, all year long. At this point we're considered to be in full deployment.

By the time we reach full deployment, we've generally tapered off the monitoring frequency to about once a month. After a few months, we can monitor even less frequently (only having someone listen to about 100 calls every 4 months). At this stage, almost all analysis is performed using statistics and automated processes. Whatever improvements we make to the system now should be minor and subtle.

Above all, we have to make sure that we're using the same metrics we used in partial deployment. The more information we have to analyze the system, the more accurate the results. If we only analyzed the calls on a single day, we might see transaction completion percentages as high as 96% or 94%. But, just as baseball players' batting averages often drop from April to July as they get more at-bats, transaction completion numbers , averaged over a long period of time, could show a drop to an average of 88%, for example. Once again, this helps the development team get a better understanding of how well the system is actually performing.

Even after a system has been deployed, it's essential to continue monitoring its accuracy and transaction completion rates. Often, changes in these numbers can help identify problems. For example, if, after a system has been up and running for six months, its transaction completion rate suddenly dropped from 98% to 86%, we would know there was a problem ” especially if the recognition accuracy rate remained high.

Here's why it can happen. Let's say a bank starts calling a particular checking account the "Active Account" as part of an advertising campaign. If callers using the system pick up that phrase from the advertisements and start using it ”for example, asking the system to transfer funds from their "Active Accounts" instead of their "Checking" or "Savings" accounts ”it's likely the system wouldn't have been programmed to recognize the phrase and, thus, would reject it.

Time enables us to see the longitudinal behavior of the system better. We can gain valuable insights that we can use to improve the system in subtle ways. That's why it's important never to consider a system completely finished. It is always a work in progress, a living thing that must adapt to change and circumstances, just as we all do in our lives. Regular, consistent monitoring over time is the only way to ensure that the people who use and depend on the system can continue to use it effectively and enjoy using it. After all, that's why we make these systems in the first place, right?