The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Points
Friday - October 07, 2005 11:47 AM in
Writing good survey questions is an art, not a science. Experienced and well-qualified practitioners can (and often do) disagree about the best possible wording for a survey question. Small changes to the way a question is worded can sometimes make big differences in the survey results, and it isn't always clear which wording was better.
One point of contention which we've encountered a lot in the past few weeks is the question of how many choices to offer on a satisfaction question. We typically use a five point scale with options ranging from "Very Satisfied" to "Very Dissatisfied," but I've seen questions with anything from a two-point scale ("Were you satisfied with the service you received? Yes or no.") to a 100-point scale ("On a scale from one to one hundred, how satisfied were you?").
All these different versions of satisfaction questions give somewhat different results, so you can't compare data gathered with a four-point question to data gathered with a seven-point question. But they all give usable information, and as far as I'm concerned, which question you use is more a matter of preference and available benchmark data than anything else.
But not everyone agrees. Recently, a highly-paid consultant made a statement more-or-less along these lines to one of our clients:
"Everyone measures caller satisfaction using a nine-point scale. If you're using a four-point scale, it's because you don't understand how to survey about caller satisfaction. I recommend you switch to a nine-point scale immediately."
Aside from the fact that it's a bad sales tactic to tell a prospect or client that they don't understand what they're doing, the statement is absurd on the face of it. But nevertheless concerned, the client called me and asked for a second opinion (my opinion was that I nearly fell out of my chair laughing). By the way, the four-point satisfaction question was not one we had written or used, but one this client had developed itself and had been using for many years.
It turned out that there was more to this story. The consultant was trying to sell an automated end-of-call survey (which we consider very bad practice, since surveying at the end of the phone call creates a very large upward bias in the data). The reason for the nine-point scale? There are nine numerical buttons on a telephone keypad if you exclude zero.
And by the way, that's not a bad reason to prefer the nine-point satisfaction question. The mechanical limitations of a particular survey method are an important consideration when designing a questionnaire.
But to represent it as the onlyway to craft a survey is simply wrong.
Posted by Peter Leppik
All these different versions of satisfaction questions give somewhat different results, so you can't compare data gathered with a four-point question to data gathered with a seven-point question. But they all give usable information, and as far as I'm concerned, which question you use is more a matter of preference and available benchmark data than anything else.
But not everyone agrees. Recently, a highly-paid consultant made a statement more-or-less along these lines to one of our clients:
"Everyone measures caller satisfaction using a nine-point scale. If you're using a four-point scale, it's because you don't understand how to survey about caller satisfaction. I recommend you switch to a nine-point scale immediately."
Aside from the fact that it's a bad sales tactic to tell a prospect or client that they don't understand what they're doing, the statement is absurd on the face of it. But nevertheless concerned, the client called me and asked for a second opinion (my opinion was that I nearly fell out of my chair laughing). By the way, the four-point satisfaction question was not one we had written or used, but one this client had developed itself and had been using for many years.
It turned out that there was more to this story. The consultant was trying to sell an automated end-of-call survey (which we consider very bad practice, since surveying at the end of the phone call creates a very large upward bias in the data). The reason for the nine-point scale? There are nine numerical buttons on a telephone keypad if you exclude zero.
And by the way, that's not a bad reason to prefer the nine-point satisfaction question. The mechanical limitations of a particular survey method are an important consideration when designing a questionnaire.
But to represent it as the onlyway to craft a survey is simply wrong.
Posted by Peter Leppik
Posted at 11:47 AM by | | | |