[…]
You’ve spent a lot of time deciding what to ask in your next market research survey, but have you considered how you want to ask a question?
Deciding on a question type, especially between a ranking scale vs. rating scale question, vexes many researchers. After all, the way a question is asked, and its format, influences your results and how you can use your data.
This post provides a ranking and rating scale definition, examples of each question type, and tips for deciding between the two common question types used when gathering consumer insights.
Recommended: What’s a Good Sample Size in Market Research?
A ranking scale forces respondents to rank a list of items with only one selection in each rank. Ranking scale questions often ask respondents to rank based on preference, but you can get creative with your ranking criteria. For instance, you can also ask people to rank new features based on importance or how likely they would buy a product.
Example 1: Rank the following snow cone flavors from 1 to 5, with 1 being “keep them coming” and 5 being the “no way.”
Example 2: Rank the following items by how much they influence your choice to buy a car, from that’s it (1), not so much (2), and don’t care (3).
If you’ve ever had the age-old debate of where to eat with a friend or significant other, you’ve probably been in a scenario where you should have used a ranking scale question. The conversation always starts innocently enough. “Hey, do you want to grab some lunch? How about tacos, sushi, or hamburgers?”
Then comes the dreaded response that starts the inevitable restaurant indecision spiral: “Oh, I don’t care. They all sound great.”
Your question did not yield any actionable data. Tacos, sushi, and hamburgers were all acceptable choices for you. You didn’t need to know if all three were acceptable to your lunch buddy. You needed to know which one was their top choice.
Use a ranking scale question when you need to know customers’ likes and dislikes compared to one another. Force-ranking questions require respondents to give some thought to their preferences. Going back to our snow cone example question.
Let’s say you’re trying to narrow down your offerings from five to three flavors. If everyone tells you they like all the flavors, that doesn’t help. But if you require them to rank them and most people rank lemon and cherry at the bottom, you know which flavors to cut from your menu.
Ranking scale questions do have some limitations. Ranking scales are not as helpful with longer lists of items. If you give people more than 5 responses, you could run into issues.
First, the longer your list, the more likely respondents are only to have extreme opinions about the best and worst options. You probably can’t gain much insight from what people put in the middle tier. Another obvious limitation of this type of question is that while it will reveal what most people prefer, it will not explain why they have that preference.
Ranking scales for long lists also can frustrate respondents, especially as more people take mobile surveys on their phones. Using drag-and-drop to reorder or entering a number next to each item is cumbersome on smaller screens. Also, if people like some things equally, they may resent being forced to rank them.
As a best practice, try to use ranking scales sparingly in any one survey. You can also try branching questions where you ask a follow-up question based on their answer. For example, you could use an open-ended text question to ask why someone ranked an item first or last.
Rating scales are one of the most frequently used question types for market research surveys. These questions ask people to pick a value to describe their level of satisfaction or agreement with something. They also can be used to measure purchase frequency.
You’ll likely recognize two of the most common types of rating scale questions: NPS (net promoter score) and Likert scales.
On a scale of one to ten, how likely are you to recommend (your brand here) to a friend or colleague?
very unlikely – 1 2 3 4 5 6 7 8 9 10 – very likely
Rate your level of agreement with the following statements:
I decide to buy a car based on the price.
Strongly agree | agree | neither agree nor disagree | disagree | strongly disagree
Rating scales help determine people’s attitudes in a way that is easy to ask and digest. Reasons to ask a ranking scale question include:
The biggest drawback to ranking scale questions is a phenomenon called satisficing. People don’t put effort into evaluating your questions and give everything the same rank because it’s easier. Remember our lunch example? Saying all three choices were fine is an example of satisficing. Results from rating scale questions are often too similar to help a company choose between multiple options.
Deciding between a ranking scale vs. rating scale question can be difficult, but when you start to understand the limitations of each question type, you’ll see there is a place for both in your research. When you need to determine general attitudes and feedback, use a rating scale. If you need to determine priority, use a ranking scale.
Here’s how you could use both questions types as a part of your product development process. Let’s assume you’re designing a new smartwatch.
To see how quickly you can start asking rating and ranking questions on our mobile platform, book a demo with one of our OnePulse research experts.
Click here to view the website in .