The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
Not All Surveys are Created Equal
Thursday - September 08, 2005 03:44 PM in
There's a conversation we have with many different people at a wide variety of companies. It goes something like this:
VocaLabs: What do you do for quality assurance in your call center?
Prospective Client: We record calls, coach agents, and survey our customers about the quality of the service they're receiving.
VL: What kind of survey do you use?
PC (suddenly looking less confident): I don't understand the question. We do a customer satisfaction survey.
VL: Right, but how do you gather the data? Do you ask people to go to a website? Do you mail them a paper survey? Do you phone them? Do you do a survey at the end of the call?
PC (enlightened): Oh, I see. We use survey method X. Does it matter?
We live and breathe survey technique here at VocaLabs, and sometimes it's easy to forget that most people don't really give it that much thought. But the type of survey you use to gather data matters a lot, and some methods work very poorly. In our book, Gourmet Customer Service, we cover a lot of different data collection techniques, and the details of the method can affect who takes the survey, what kinds of answers they give, how long it takes to get the results, and of course cost.
For example, considerations for different survey methods include:
And those are just the considerations for the type of survey you choose. We haven't even gotten into issues like how do you write the survey questions (more of an art than a science), and how do you interpret the results.
So what's the typical manager to do, short of becoming an expert in survey methodology? Just like everything else, the best thing to do is find an expert and rely on him or her. And how do you find an expert? Here are some rules of thumb:
Posted by Peter Leppik
Prospective Client: We record calls, coach agents, and survey our customers about the quality of the service they're receiving.
VL: What kind of survey do you use?
PC (suddenly looking less confident): I don't understand the question. We do a customer satisfaction survey.
VL: Right, but how do you gather the data? Do you ask people to go to a website? Do you mail them a paper survey? Do you phone them? Do you do a survey at the end of the call?
PC (enlightened): Oh, I see. We use survey method X. Does it matter?
We live and breathe survey technique here at VocaLabs, and sometimes it's easy to forget that most people don't really give it that much thought. But the type of survey you use to gather data matters a lot, and some methods work very poorly. In our book, Gourmet Customer Service, we cover a lot of different data collection techniques, and the details of the method can affect who takes the survey, what kinds of answers they give, how long it takes to get the results, and of course cost.
For example, considerations for different survey methods include:
- Online surveys are very cost-effective, and often gather a lot of useful feedback. But the response rate is very low unless you're directing people to the survey from another web site. That's because most people can't be bothered to write down a URL during a phone call, and won't remember to go to the webpage.
- Mail surveys are also cost-effective, and can get a good response rate. But they are very slow: the customer won't even get the survey until days after it is sent out, and it can take weeks to get the response. If you're trying to survey about a particular experience (for example, the call to customer service someone made Tuesday afternoon), most people will have forgotten the details of the call by the time they get the survey. Worse, they may get confused and fill out the survey based on a different call.
- Calling a customer back to take a survey after a customer service call--also known as a follow-up survey--can get good data, but this method is relatively expensive. Also, because of technical limitations, most survey operations don't call the customer back until 24 to 72 hours after the initial call, by which time the customer may have forgotten what happened. In addition, because of the wariness we all have developed towards telemarketers, response rates tend to be low. (It doesn't have to be that way, though. We'll be making an exciting announcement in a few days about a new survey technique we've developed. Watch this space.)
- Doing an automated survey before the customer hangs up--an end-of-call survey--sounds like a great idea, but in practice it is one of the worst ways to collect data. The problem is that callers who hang up before they get to the survey are excluded from the data. And the people most likely to hang up prematurely are exactly the people you most want feedback from: people who got frustrated, angry, lost, or trapped in an IVR. End-of-call surveys will always make it look like you're doing better than you really are, and in the worst cases they can make it appear that things are improving when they're really getting worse.
And those are just the considerations for the type of survey you choose. We haven't even gotten into issues like how do you write the survey questions (more of an art than a science), and how do you interpret the results.
So what's the typical manager to do, short of becoming an expert in survey methodology? Just like everything else, the best thing to do is find an expert and rely on him or her. And how do you find an expert? Here are some rules of thumb:
- A true professional may have a favorite survey technique, but will acknowledge that every method has its shortcomings, and will help you choose the best method for your particular needs.
- A true professional will understand the sources of bias and error in a survey and work to mitigate them. It is not necessary (or desirable) to correct for all biases--especially since the correction itself can introduce new errors--but the biases should be acknowledged.
- A true professional will give you the tools to understand the data and put it in context. Just saying that you have a 79% satisfaction rating doesn't help you a lot. You should be given the tools to understand whether that score is good or bad (based on your goals), and how it can be improved.
- A true professional understands that surveying is an art, not a science. Be very wary of anyone who claims to have all the answers, or a single solution for any problem.
Posted by Peter Leppik
Posted at 03:44 PM by | | | |