The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
The Art of Writing a Survey
Monday - February 06, 2006 02:06 PM in
Last week at SpeechTEK West, I participated in a group discussion headed by Susan Hura about how to survey callers to get information about how well a speech recognition system is performing.
It was a productive and interesting session, and there's going to be an article in Speech Technology magazine coming out of our efforts, chock full of useful tips for how to create and administer a survey.
But try as we might, we could not agree on a set of specific survey questions we could all get behind for measuring the performance of speech applications across the board. We can all agree that, for example, you should avoid leading questions and be as consistent as possible in how you administer the survey. Throw out a specific question, though, and all heck breaks loose.
How many different choices should a customer have on a satisfaction question? Five? Seven? Or ten? And what if the client insists on four? Should you label every choice ("Very Satisfied," "Somewhat Satisfied," "Neutral," "Somewhat Dissatisfied," or "Very Dissatisfied")? Or just the end points ("On a scale of one to five, with one being Very Satisfied and five being Very Dissatisfied....")?
And how should the question be worded? Should it be "How satisfied were you...." or "How satisfied or dissatisfied were you..."?
With eight people in the room, all of us surveying professionals, we could hardly get more than three people to agree on anythingabout a specific survey question, no matter how trivial.
The reason is that all of these choices give perfectly reasonable survey questions, but different answers. So everyone in the room uses a particular set of survey questions which they've used over and over, and have a lot of experience and data about. It is more important to be consistent in the survey questions you use than it is to find The Perfect Satisfaction Question, so if we endorsed a particular form for a (for example) satisfaction question, then everyone would have to essentially start over building data and experience with that particular question.
This is why I always say that writing a survey is an art, not a science. It is better to find a set of questions which works for you and stick with it, than to keep tweaking and trying to make it better.
Posted by Peter Leppik
But try as we might, we could not agree on a set of specific survey questions we could all get behind for measuring the performance of speech applications across the board. We can all agree that, for example, you should avoid leading questions and be as consistent as possible in how you administer the survey. Throw out a specific question, though, and all heck breaks loose.
How many different choices should a customer have on a satisfaction question? Five? Seven? Or ten? And what if the client insists on four? Should you label every choice ("Very Satisfied," "Somewhat Satisfied," "Neutral," "Somewhat Dissatisfied," or "Very Dissatisfied")? Or just the end points ("On a scale of one to five, with one being Very Satisfied and five being Very Dissatisfied....")?
And how should the question be worded? Should it be "How satisfied were you...." or "How satisfied or dissatisfied were you..."?
With eight people in the room, all of us surveying professionals, we could hardly get more than three people to agree on anythingabout a specific survey question, no matter how trivial.
The reason is that all of these choices give perfectly reasonable survey questions, but different answers. So everyone in the room uses a particular set of survey questions which they've used over and over, and have a lot of experience and data about. It is more important to be consistent in the survey questions you use than it is to find The Perfect Satisfaction Question, so if we endorsed a particular form for a (for example) satisfaction question, then everyone would have to essentially start over building data and experience with that particular question.
This is why I always say that writing a survey is an art, not a science. It is better to find a set of questions which works for you and stick with it, than to keep tweaking and trying to make it better.
Posted by Peter Leppik
Posted at 02:06 PM by | | | |