The Customer Service Survey
Shifting Testing Attitudes
Thu - February 2, 2006 02:07 PM in
Peter and I are just back from two weeks of trade show participation, one in Miami and another in San Francisco. I've mentioned before that it is gratifying to see the call center industry has moved from a position of testing self service customer service technology as an afterthought to a dawning realization of the importance of making sure a design works well before subjecting actual customers to the service.
This shift was noticed again when after a talk I gave, the majority of the Q/A surrounded "How much should I budget for testing?" No longer, why test or how to test, this shift to how much to spend was a great sign. But there was still a touch of confusion surrounding the definition of a "usability" test and what different type tests are necessary.
During the early development sequence of a self service customer care application, a Voice User Interface (VUI) designer lays out the voice prompts the caller is to answer to, and tests to see if the design is workable. In the past this was often accomplished by what is known as a Wizard of Oz test in which a test participant is observed while a test administrator (the wizard behind the curtain) plays or reads the voice prompts. Today, the use of a wizard is being supplanted by the ability to easily create a working prototype on which to test, but the strategy is the same.
To some, this laboratory setting using but a few subjects is the definition of a usability test, and was the only testing of workability and caller ease of use that was performed on a design. Experience has taught us that while the lab test remains important, what it really showed was how well the application worked for just a few individuals and falls short of knowing if a design will serve a large group of users. Thus a number of mediocre designs were released and is a big reason for the bad rap self service customer care has received. Our mantra, and increasingly the realization of others is that "usability" includes any item whether design or technology that impacts the caller experience, and that the only way to know if a design will give good service is to test it with hundreds of typical callers.
These larger scale tests are Vocalabs forte and is why we can make statements about the accuracy and low margin of error of a Vocalabs usability study. But whether you subscribe to the laboratory, VocaLabs or combination of both approach, it is not enough. To be thorough, an automated application also has to test for how the system performs when subjected to a load of simultaneous calls (a stress or load test). And as an application increases in complexity, the traversal test in which functionality of every prompt and every likely response in a variety of call paths, also becomes more critical.
Posted by Rick Rappe'
During the early development sequence of a self service customer care application, a Voice User Interface (VUI) designer lays out the voice prompts the caller is to answer to, and tests to see if the design is workable. In the past this was often accomplished by what is known as a Wizard of Oz test in which a test participant is observed while a test administrator (the wizard behind the curtain) plays or reads the voice prompts. Today, the use of a wizard is being supplanted by the ability to easily create a working prototype on which to test, but the strategy is the same.
To some, this laboratory setting using but a few subjects is the definition of a usability test, and was the only testing of workability and caller ease of use that was performed on a design. Experience has taught us that while the lab test remains important, what it really showed was how well the application worked for just a few individuals and falls short of knowing if a design will serve a large group of users. Thus a number of mediocre designs were released and is a big reason for the bad rap self service customer care has received. Our mantra, and increasingly the realization of others is that "usability" includes any item whether design or technology that impacts the caller experience, and that the only way to know if a design will give good service is to test it with hundreds of typical callers.
These larger scale tests are Vocalabs forte and is why we can make statements about the accuracy and low margin of error of a Vocalabs usability study. But whether you subscribe to the laboratory, VocaLabs or combination of both approach, it is not enough. To be thorough, an automated application also has to test for how the system performs when subjected to a load of simultaneous calls (a stress or load test). And as an application increases in complexity, the traversal test in which functionality of every prompt and every likely response in a variety of call paths, also becomes more critical.
Posted by Rick Rappe'
Posted at 02:07 PM | | | | |