The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
What percentage of calls should I survey?
Thursday - March 23, 2006 03:05 PM in
Dear Survey Doctor,
Help! I've never done satisfaction surveys in my call center before, and I think I'm ready to do it for the first time. But the consultant I hired has really been pressuring me. He says that in order to get good data I need to survey at least 5% of all my callers. I don't think I'm ready for that! What percentage of callers should I survey to make sure I'm getting a statistically meaningful sample? --Confused in Chattanooga
Help! I've never done satisfaction surveys in my call center before, and I think I'm ready to do it for the first time. But the consultant I hired has really been pressuring me. He says that in order to get good data I need to survey at least 5% of all my callers. I don't think I'm ready for that! What percentage of callers should I survey to make sure I'm getting a statistically meaningful sample? --Confused in Chattanooga
Dear Confused,
Don't be embarrassed: this is a common question among call center managers who are thinking of surveying for the first time. It seems that everyone talks about survey methods, statistical relevance, and sample bias, but nobody ever tells you what percentage of calls to survey.
Ask any statistician, and you'll always get the same answer: percentage doesn't matter. What matters is the numberof surveys you do, not the percentage of calls. Don't let the consultant trick you into doing more surveys than you need by making you feel insecure about the data.
Through the confusing magic of statistics, it turns out that surveying 1,000 out of 100,000 callers (1% of the callers) is just as good as surveying 1,000 out of 10,000,000 callers (0.01% of the callers). In either case, the margin of error will be just a little over 3 percentage points.
What matters much more is how you choose the 1,000 people to survey. To get the best possible results, you need to choose 1,000 people completely at random. It turns out that's really hard to do, since you can't just pick 1,000 people and force them to take the survey. Some people are more likely to take a survey than others, and sometimes the way you pick people can make a big difference (this is called sample bias, and lately I've been writing about one really bad method, doing surveys at the end of a call).
So what you really should do is figure out what margin of error you can live with and how you want to slice the data (for example, do you want a 3% margin of error when you look at each skill set individually?), and figure out how many surveys you need to get there. Forget the percentage of calls.
And you might also want to consider whether this consultant is really the one you want to work with. Does he really care about what's right for you, or does he just want to get into your wallet? --Survey Doctor
Posted by Peter Leppik
Don't be embarrassed: this is a common question among call center managers who are thinking of surveying for the first time. It seems that everyone talks about survey methods, statistical relevance, and sample bias, but nobody ever tells you what percentage of calls to survey.
Ask any statistician, and you'll always get the same answer: percentage doesn't matter. What matters is the numberof surveys you do, not the percentage of calls. Don't let the consultant trick you into doing more surveys than you need by making you feel insecure about the data.
Through the confusing magic of statistics, it turns out that surveying 1,000 out of 100,000 callers (1% of the callers) is just as good as surveying 1,000 out of 10,000,000 callers (0.01% of the callers). In either case, the margin of error will be just a little over 3 percentage points.
What matters much more is how you choose the 1,000 people to survey. To get the best possible results, you need to choose 1,000 people completely at random. It turns out that's really hard to do, since you can't just pick 1,000 people and force them to take the survey. Some people are more likely to take a survey than others, and sometimes the way you pick people can make a big difference (this is called sample bias, and lately I've been writing about one really bad method, doing surveys at the end of a call).
So what you really should do is figure out what margin of error you can live with and how you want to slice the data (for example, do you want a 3% margin of error when you look at each skill set individually?), and figure out how many surveys you need to get there. Forget the percentage of calls.
And you might also want to consider whether this consultant is really the one you want to work with. Does he really care about what's right for you, or does he just want to get into your wallet? --Survey Doctor
Posted by Peter Leppik
Posted at 03:05 PM by | | | |