The Customer Service Survey
What's Wrong with the Rank Order Question?
Tue - October 16, 2007 02:55 PM in
Time for a little descent into survey geekery, with a short rant about the rank order question.
What's a rank order question? It's a question where the survey participant is asked to rank a number of different choices from best to worst, for example:
Please rank these kinds of vegetables from 1 (favorite) to 4 (least favorite):
____ Carrots
____ Celery
____ Artichokes
____ Arugula
Rank order questions don't appear very often on customer service related surveys, but you do see them from time to time in market research. So why don't I like them?
1) They're more confusing than other kinds of survey questions.
2) They force the participant to make a choice even if s/he doesn't have an opinion (most rank order questions don't allow the participant to specify a "tie"), which can lead to the illusion of stronger data than what really exists.
3) They only work in writing: it is difficult to craft a rank order question which doesn't bog down hopelessly in an interview or (even worse!) an IVR survey.
4) A rank-order question actually provides lessdata than a series of Likert-scale questions (for example: How much do you like carrots? A lot / somewhat / a little / not at all), and the Likert-scale questions are easier to understand, quicker to complete, and translate better into different media.
So next time you're tempted to write a rank order question, try writing it as a series of Likert-scale questions instead. You'll probably be happier with the results.
Posted by Peter Leppik
Please rank these kinds of vegetables from 1 (favorite) to 4 (least favorite):
____ Carrots
____ Celery
____ Artichokes
____ Arugula
Rank order questions don't appear very often on customer service related surveys, but you do see them from time to time in market research. So why don't I like them?
1) They're more confusing than other kinds of survey questions.
2) They force the participant to make a choice even if s/he doesn't have an opinion (most rank order questions don't allow the participant to specify a "tie"), which can lead to the illusion of stronger data than what really exists.
3) They only work in writing: it is difficult to craft a rank order question which doesn't bog down hopelessly in an interview or (even worse!) an IVR survey.
4) A rank-order question actually provides lessdata than a series of Likert-scale questions (for example: How much do you like carrots? A lot / somewhat / a little / not at all), and the Likert-scale questions are easier to understand, quicker to complete, and translate better into different media.
So next time you're tempted to write a rank order question, try writing it as a series of Likert-scale questions instead. You'll probably be happier with the results.
Posted by Peter Leppik
Posted at 02:55 PM | | | | |