The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
I Can't Remember, but Give Me a Minute and I'll Come Up With Something.
Friday - September 09, 2005 11:54 AM in
My last few blog entries have been directed at obstacles that make administering a survey with reliable results more difficult than most realize. And today's entry has to do with yet another major barrier to receiving reliable survey results.
Not only is it important to survey a large enough sample that the results have a low margin of error; it is also important to get feedback before detailed memory of the contact is lost.
Humans forget 95% of what they hear within 72 hours. I have been unable to find the original source for this often quoted statistic, but the implication is more important than a precise number anyway. The greater the time span between a customer interaction and questions about that interaction, the less detailed and more generic the questions become because participants simply can't remember enough detail to give reliable answers to more penetrating questions.
The consequences of "brain fade" add still another worrisome twist. Rather than admit they can't remember, study participants are prone to give just any answer in order to get the survey over with. We even occasionally encounter a caller experience report in which the respondent gives plenty of detailed info, usually negative. Yet when we listen to a call recording of the contact; we find that what the participant reports never actually happened!
Are these survey respondents lying? Technically I suppose so, but my sense is that some variation of deja vu causes the participant to unconsciously fill in the lost memory with a retained memory of another experience.
Traditional methods of gathering feedback often make large enough and fast enough so resource intense and expensive that lesser methods evolved out of economic necessity. A result is that even today, very many business decisions are made with insufficient or poor study data incorrectly believed to be accurate.
All my category of problem explanations I've written about can be summed up this way: Since humans resist survey taking, in order to control sample and participation bias, care is needed to insure that those that do participate are representative of the group from which you are seeking answers. "Satisfied" is not a positive response and phrasing questions objectively requires some expertise. The survey must be administered without bias from those asking the questions. Size matters. Larger samples are needed to reduce the margin of error to acceptable levels. Speed matters in order to minimize the human memory loss problem. Bad data leads to bad decisions. Whether the reason for conducting studies with these flaws was due to cost limitations, or the result of just not knowing better; poor survey technique is common and usually found with more than one of these problems in some overlapping configuration.
Posted by Rick Rappe
Humans forget 95% of what they hear within 72 hours. I have been unable to find the original source for this often quoted statistic, but the implication is more important than a precise number anyway. The greater the time span between a customer interaction and questions about that interaction, the less detailed and more generic the questions become because participants simply can't remember enough detail to give reliable answers to more penetrating questions.
The consequences of "brain fade" add still another worrisome twist. Rather than admit they can't remember, study participants are prone to give just any answer in order to get the survey over with. We even occasionally encounter a caller experience report in which the respondent gives plenty of detailed info, usually negative. Yet when we listen to a call recording of the contact; we find that what the participant reports never actually happened!
Are these survey respondents lying? Technically I suppose so, but my sense is that some variation of deja vu causes the participant to unconsciously fill in the lost memory with a retained memory of another experience.
Traditional methods of gathering feedback often make large enough and fast enough so resource intense and expensive that lesser methods evolved out of economic necessity. A result is that even today, very many business decisions are made with insufficient or poor study data incorrectly believed to be accurate.
All my category of problem explanations I've written about can be summed up this way: Since humans resist survey taking, in order to control sample and participation bias, care is needed to insure that those that do participate are representative of the group from which you are seeking answers. "Satisfied" is not a positive response and phrasing questions objectively requires some expertise. The survey must be administered without bias from those asking the questions. Size matters. Larger samples are needed to reduce the margin of error to acceptable levels. Speed matters in order to minimize the human memory loss problem. Bad data leads to bad decisions. Whether the reason for conducting studies with these flaws was due to cost limitations, or the result of just not knowing better; poor survey technique is common and usually found with more than one of these problems in some overlapping configuration.
Posted by Rick Rappe
Posted at 11:54 AM by | | | |