Ranking question definition
Want to know what aspects of your product the respondents like best? List them and let your respondents rank the items themselves by asking a ranking question. This allows survey respondents to compare different items to each other by placing them in order of how they score (or rank) at a specific aspect, such as design, cost, functionality, importance. Often with the most important or preferred item ranked first or on top of the list.
Ranking vs. Rating questions
Example of a ranking question wording: Please rank the following options in order of importance, with the most important item on top.
Example of a rating question wording: To what extent do you agree with the following statement? (Strongly agree/agree/unsure/disagree/strongly disagree)
Ranking questions are different from rating questions which ask respondents to rate items or statements and the respondent could choose the same response, e.g. “always” or “strongly disagree” for a number of options. In contrast, in a ranking question, the respondent must make comparisons to all of the answers, and decide how they want to order them based on their relative importance, priority, intensity, etc.
When deciding whether it is best to use a ranking question or a rating question, think about what you will be trying to analyze. If you want to be able to compare between different items, using a ranking question will give useful insight into how respondents make comparisons. However, if you want to use actual values in analysis, it will be more helpful to use a rating question, and for example, asking respondents to rate the importance of an item on a scale of 1 to 10.
Ranking question example
The example below uses a ranking scale of #1 to #4, with #1 being the most important object to #4 being the least important object.
XY drag and drop question
One variant of the classic ranking question is the XY drag and drop question. This allows the user to rank items on two characteristics simultaneously.
- Make sure that the list of items you are asking the respondent to rank are actually all part of the same scale, or can be judged using the same criteria.
- The longer the list of items presented to respondents, the harder it becomes for them to accurately rank them. Think carefully about presenting more than five options in your ranking question. If you have a long list such as ten options that you definitely want to present, it may be more effective to request that the respondent chooses and ranks their top five priorities, concerns, etc. Note that this will result in more work at the data analysis stage.
- Remember that when using a ranking question, you are forcing respondents to judge the items, and when they choose to rank one item higher on the list, other items will be automatically ranked lower. Make sure that the items you are asking your respondents to judge are actually fair to be judged in relation to each other.