The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Why We Do What We Do
Tuesday - August 23, 2005 06:33 PM in
Those of you who've heard my sales presentation know that I regularly explain that Peter was an investment analyst specializing in tracking companies that provide call center technology. I also mention that when he set out to find why amazing advances in technology were not resulting in consumer reports of improved service, he discovered that traditional methods of gauging ease of use and caller satisfaction simply aren't too reliable.
Companies that want to learn more about their customers often use a method generically called the follow up survey. Basically, to find out what customers think, you just ask them. There are variations on how and when to ask and what medium to use such as phone, mail, bill inserts and more recently on-line questionnaires and surveys; but it seems simple, right?
The trouble is that survey administration is not simple at all. Ignoring question and questioner bias for now, and focusing just on the problems related to customer participation, here's what we find.
Consumers don't like taking surveys. They resent the imposition on their time, don't believe the company is serious about what they think, and are reluctant to express negative opinions either because they don't want to be confrontational or out of a fear that negative feedback will flag them as a trouble account and make future dealings harder. The result is low participation and sample bias because only those who have strong opinions (usually negative) have the motivation to participate. It isn't all a bad thing to have more pessimistic feedback if the goal is to get your arms around where to focus on improving. But all you really know is that the overall outcome of typical follow up studies are wrong. You just don't know what, where or how far off the results are.
Problem two is that size matters. For example, if you survey 100 customers each month, and find that satisfaction scores improved by perhaps 5%, you may be tempted to pass out bonuses. But since a 100 participant study at best has a 10% margin of error, you don't know if you actually improved.
Then we add the problem of brain fade. Humans are said to begin to forget details of what they hear within hours, and after three days can only recall 5% of what they heard. One consequence is that you are stuck asking more generic questions, and even then, there is the human tendency to want to please. A result can be that people sometimes make up answers rather than admit they can't recall specifics. In the extreme, we see instances where listening to call recordings shows us that what the caller reports, never even happened!
The plain fact is that for any follow up study to improve accuracy and get closer to the truth, the study has to be large enough to minimize margin of error and participation bias and fast enough so that questions can be asked before memory loss becomes significant.
But until Peter's solution became Vocal Laboratories Inc., big enough and fast enough often meant too expensive to be affordable. Companies settled for smaller samples and slower response methods, and the lack of accurate results is either ignored or not understood.
One consequence is the mediocre service we all receive because in many instances companies have deluded themselves into believing they are doing a better job than they really are.
Posted by Rick Rappe
The trouble is that survey administration is not simple at all. Ignoring question and questioner bias for now, and focusing just on the problems related to customer participation, here's what we find.
Consumers don't like taking surveys. They resent the imposition on their time, don't believe the company is serious about what they think, and are reluctant to express negative opinions either because they don't want to be confrontational or out of a fear that negative feedback will flag them as a trouble account and make future dealings harder. The result is low participation and sample bias because only those who have strong opinions (usually negative) have the motivation to participate. It isn't all a bad thing to have more pessimistic feedback if the goal is to get your arms around where to focus on improving. But all you really know is that the overall outcome of typical follow up studies are wrong. You just don't know what, where or how far off the results are.
Problem two is that size matters. For example, if you survey 100 customers each month, and find that satisfaction scores improved by perhaps 5%, you may be tempted to pass out bonuses. But since a 100 participant study at best has a 10% margin of error, you don't know if you actually improved.
Then we add the problem of brain fade. Humans are said to begin to forget details of what they hear within hours, and after three days can only recall 5% of what they heard. One consequence is that you are stuck asking more generic questions, and even then, there is the human tendency to want to please. A result can be that people sometimes make up answers rather than admit they can't recall specifics. In the extreme, we see instances where listening to call recordings shows us that what the caller reports, never even happened!
The plain fact is that for any follow up study to improve accuracy and get closer to the truth, the study has to be large enough to minimize margin of error and participation bias and fast enough so that questions can be asked before memory loss becomes significant.
But until Peter's solution became Vocal Laboratories Inc., big enough and fast enough often meant too expensive to be affordable. Companies settled for smaller samples and slower response methods, and the lack of accurate results is either ignored or not understood.
One consequence is the mediocre service we all receive because in many instances companies have deluded themselves into believing they are doing a better job than they really are.
Posted by Rick Rappe
Posted at 06:33 PM by | | | |