The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Just how satisfied can you get?
Thursday - September 29, 2005 07:08 PM in
I was at a SOCAP (Society of Consumer Affairs Professionals) local chapter breakfast meeting this AM where Scott Broetzmann, president of Customer Care Measurement and Consulting, was the guest speaker. He discussed a study commissioned by SOCAP (and for sale to members, so Scott could not give a full report), and used this data as a launching point to talk about changed consumer opinions about customer service. It was gratifying to see result after result match up with Vocalabs findings.
But there is one item on which we might diverge, and which I'll need to speak about in generalities as I am far from an analytical statistician.
One of our clients recently asked why we use a five point scale to rate caller satisfaction, when a "consultant" was telling them that nine points is the standard that "everybody" uses.
The simple answer that I was ready to accept is because there are nine usable buttons on a touchtone phone making a 1-9 scale possible. But is it practical? The answer was explained to me as no it isn't. Now, both David and Peter Leppik explained to me in great detail the theory behind a Likert scale, and added a whole bunch of mathematical points some of which that I actually understood.
But Scott was using a finer granulation on the positive side of the scale and so I asked the question of, where the disconnect was since I see companies reporting 70-97% "satisfaction" scores and he was showing much smaller average numbers. His answer was basically that companies fudge the data by adding all the scores together, and they shouldn't because there was such a distinct difference between levels of satisfaction and other metrics such as brand loyalty. In other words, he was saying that a really really satisfied caller was so much more loyal than a sort of satisfied one, that a finer breakdown was important.
I shared this with Peter and David, who promptly shot my report full of holes. The gist is this. Satisfaction is a fuzzy term anyway meaning different things to different people, so too much precision isn't practical. They also explained human nature in that people don't know whether they were 67% satisfied or 82.5% satisfied...whatever; so the result is that they ignore the granularity and choose basic very dissatisfied, dissatisfied, satisfied (neutral), somewhat satisfied and very satisfied anyway.
Further that while what Scott was saying is true, that the more satisfied a caller is, the more loyal they are, breaking it down much finer than this is just not statistically reliable.
I guess the moral of the story is not to get overly hung up on trying to statistically correlate satisfaction scores to anything more than getting more responses on the positive side than the negative side. The finer you try to segregate happy versus unhappy callers, the more likely your data is suspect.
Posted by Rick Rappe
One of our clients recently asked why we use a five point scale to rate caller satisfaction, when a "consultant" was telling them that nine points is the standard that "everybody" uses.
The simple answer that I was ready to accept is because there are nine usable buttons on a touchtone phone making a 1-9 scale possible. But is it practical? The answer was explained to me as no it isn't. Now, both David and Peter Leppik explained to me in great detail the theory behind a Likert scale, and added a whole bunch of mathematical points some of which that I actually understood.
But Scott was using a finer granulation on the positive side of the scale and so I asked the question of, where the disconnect was since I see companies reporting 70-97% "satisfaction" scores and he was showing much smaller average numbers. His answer was basically that companies fudge the data by adding all the scores together, and they shouldn't because there was such a distinct difference between levels of satisfaction and other metrics such as brand loyalty. In other words, he was saying that a really really satisfied caller was so much more loyal than a sort of satisfied one, that a finer breakdown was important.
I shared this with Peter and David, who promptly shot my report full of holes. The gist is this. Satisfaction is a fuzzy term anyway meaning different things to different people, so too much precision isn't practical. They also explained human nature in that people don't know whether they were 67% satisfied or 82.5% satisfied...whatever; so the result is that they ignore the granularity and choose basic very dissatisfied, dissatisfied, satisfied (neutral), somewhat satisfied and very satisfied anyway.
Further that while what Scott was saying is true, that the more satisfied a caller is, the more loyal they are, breaking it down much finer than this is just not statistically reliable.
I guess the moral of the story is not to get overly hung up on trying to statistically correlate satisfaction scores to anything more than getting more responses on the positive side than the negative side. The finer you try to segregate happy versus unhappy callers, the more likely your data is suspect.
Posted by Rick Rappe
Posted at 07:08 PM by | | | |