The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Size Matters
Thursday - September 01, 2005 10:52 AM in
In an earlier blog entry I focused on the bias problems due to customer resistance to survey participation, and made passing mention that so long as the sample group being surveyed is representative of typical callers, sample size wasn't critical. True. But for reasons I can only assume are cost and perhaps a lack of understanding, we frequently encounter studies that are simply too small to rely on.
For example, if your customer satisfaction rating improved perhaps 5% since the last measurement, and you based the study on 100 customers; you cannot say with any certainty that you have improved since the margin of error in a 100 person survey is 10% (or more if the survey has other problems).
Statistically, if one caller in twenty encounters a particular problem (5% of callers or 100% of callers 5% of the time), 100 test calls or feedback reports will reliably hit the problem at least one time (99.4% probability). And if hit only once, will it be recognized as a problem? It takes 500 surveys to hit a 1% problem with the same probability, and in volume, even 1% can mean a significant number of unhappy callers.
Next, consider that people call for differing reasons regardless of whether they need a live agent or are transacting business via self-service technology. So with even 500 participants, if feedback is spread over too many reasons for calling, the odds are high that significant issues will still be missed.
Posted by Rick Rappe
Statistically, if one caller in twenty encounters a particular problem (5% of callers or 100% of callers 5% of the time), 100 test calls or feedback reports will reliably hit the problem at least one time (99.4% probability). And if hit only once, will it be recognized as a problem? It takes 500 surveys to hit a 1% problem with the same probability, and in volume, even 1% can mean a significant number of unhappy callers.
Next, consider that people call for differing reasons regardless of whether they need a live agent or are transacting business via self-service technology. So with even 500 participants, if feedback is spread over too many reasons for calling, the odds are high that significant issues will still be missed.
Posted by Rick Rappe
Posted at 10:52 AM by | | | |