The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Selection Bias
Wednesday - August 31, 2005 12:27 PM in
Traditional follow up surveys in which the customer is contacted at some later date; and the more recent end-of-call survey method in which the caller chooses to stay on the line to take a (usually automated) survey, have a variety of shortcomings that need to be understood.
It is more difficult than most realize to create a truly objective set of survey questions (If you ask "Do you like milk?" the answers are different if the prior question was "Do you like cookies?"); but for the purpose of Part One, the focus is on biased survey participation.
Low participation numbers is the first problem that many think of. But unto itself, if the sample you survey is representative of the entire group you want answers from; low numbers may not be a critical issue. But getting a representative sample isn't easy. Plainly said, only those who have strong opinions are motivated to participate. The result is an unknown degree of sample bias.
And the sample/participation bias problem is somewhat different with a follow up survey than with an end-of-call version. With a follow up survey, all categories of customer from happy to angry don't want to be bothered. But with an end-of-call study, the bias tends to be greater since the very dissatisfied usually hang up before getting to the survey (either out of frustration with the contact, or an employee failure to transfer unhappy callers to the survey). So while the end-of-call study often garners greater participation numbers, the skew is toward over participation by more satisfied callers and you receive insufficient answers from the very callers you need to hear from the most. The result is that end-of-call studies tend to be even less accurate than the follow up survey since follow up study participation bias is more evenly spread over both the happy and the unhappy caller.
All you know for sure is the results of either survey method are wrong, just not how wrong.
Posted by Rick Rappe
Low participation numbers is the first problem that many think of. But unto itself, if the sample you survey is representative of the entire group you want answers from; low numbers may not be a critical issue. But getting a representative sample isn't easy. Plainly said, only those who have strong opinions are motivated to participate. The result is an unknown degree of sample bias.
And the sample/participation bias problem is somewhat different with a follow up survey than with an end-of-call version. With a follow up survey, all categories of customer from happy to angry don't want to be bothered. But with an end-of-call study, the bias tends to be greater since the very dissatisfied usually hang up before getting to the survey (either out of frustration with the contact, or an employee failure to transfer unhappy callers to the survey). So while the end-of-call study often garners greater participation numbers, the skew is toward over participation by more satisfied callers and you receive insufficient answers from the very callers you need to hear from the most. The result is that end-of-call studies tend to be even less accurate than the follow up survey since follow up study participation bias is more evenly spread over both the happy and the unhappy caller.
All you know for sure is the results of either survey method are wrong, just not how wrong.
Posted by Rick Rappe
Posted at 12:27 PM by | | | |