SUPR-Q: Standardized User Experience Percentile Rank Questionnaire

What you’ll find in this article:

• What is SUPR-Q?
• How does SUPR-Q work?
• When to Use SUPR-Q 
• How to get the most out of SUPR-Q 
• Pros and Cons of SUPR-Q 
• Using SUPR-Q with Qualaroo 
• Conclusion

What is SUPR-Q? 

SUPR-Q: Standardized User Experience Percentile Rank Questionnaire

SUPR-Q is a validated way to measure the general user experience on a website or application. It includes 8 questions covering 4 different areas: usability, trust/credibility, loyalty, and appearance. Loyalty is measured in part by using NPS. The SUPR-Q was developed in 2015 by Jeff Sauro of measuringU. (Source)

How does SUPR-Q work?

Like many usability questionnaires, the SUPR-Q is not intended to be diagnostic or to be a clear indicator of the entire user experience despite its name. Instead it helps to assess the usability, credibility and user perceptions of loyalty and appearance. In particular, the SUPR-Q is seen as a valuable tool for analyzing websites, even though it can be applied to applications as well. 

To get the most value out of the SUPR-Q, it’s recommended that you launch this 8-item questionnaire to users while they are experiencing your website as an intercept or in-app survey. Alternatively, you could also ask users to answer these questions after they’ve completed all tasks in a usability study. Waiting any longer than this can impact the quality of user responses as their experience of your site will not be as fresh in their mind. 

The Questionnaire

This 8 item questionnaire is put on a 5-point Likert scale from Strongly Disagree to Strongly Agree. One question though, the standard NPS question “How likely are you to recommend…?” should stay on its 11-point scale of 0-10. Here is the questionnaire below, broken into the four different dimensions that it’s designed to cover. 

Usability

1. The website is easy to use.

2. It is easy to navigate within the website. 

Credibility 

3. I feel comfortable purchasing from the website. 

4. I feel confident conducting business on the website. 

Loyalty

5. How likely are you to recommend this website to a friend or colleague?

6. I will likely return to the website in the future. 

Appearance

7. I find the website to be attractive.

8. The website has a clean and simple presentation.

If you are already running an NPS campaign on your website, then you can opt for a version of the SUPR-Q template which does not include NPS by taking out question 5. 

Score Calculation 

To score your SUPR-Q results, simply average all of your scores, excluding the ‘likely to recommend’ item. For the NPS question, divide your score by 2 to convert it to the same Likert scale results as the rest of the survey. The one exception is that this number will introduce  non-integers into your data set. However, since you will be averaging the results anyway, this will not affect score interpretation in any way. 

SUPR-Q score =  ((sum of responses excluding NPS)/7) + (NPS response/2) 

Again if you are running a version of SUPR-Q that is working off of existing NPS data, it will be helpful to have the original information on hand so that you can work backwards to calculate what the average response was. Your actual Net Promoter Score itself will not be sufficient enough to help you calculate your SUPR-Q score. 

What your score means 

As you may have noticed, the SUPR-Q is explicitly named a percentile rank questionnaire. That’s because with percentile ranks, your score actually converts to a percentile. For example, a score of 50 is average. This means that a score of 50 would indicate your website is better than half of all other websites yet also not as strong as the top half of all other websites. 

Because many companies are now being mindful of ways to improve the user experience of their websites, these results may not stay fixed in this way. In other words, as the UX of websites becomes a higher quality experience, 50 may not remain the average score. 

For this reason, the creators of the SUPR-Q host a normalized database that you can use in order to compare your score to other similar websites. Unfortunately, this database is only made accessible to those who choose to purchase the full SUPR-Q license which can charge as much as $5,000 per year.  

When to Use SUPR-Q 

Again, while the SUPR-Q can be used across multiple devices, it is most strongly recommended for assessing web or mobile web design. The SUPR-Q will prove most useful during the validation phase as you seek to make improvements and test out how users react to your digital product. 

For that reason, the SUPR-Q is something that you would likely want to run at strategic points in a product’s development rather than as a continuous in-app survey on your website. 

Here are some instances when the SUPR-Q might prove particularly useful to you. 

Redesigning your website 

If your organization has recently decided that it's time to refresh your digital presence, the SUPR-Q can allow you to do so with greater intentionality. Everyone wants their website to feel more visually appealing and user-friendly. But if you’re not measuring the before and after metrics of the changes you make to your website, you’re left shooting in the dark as to how successful your redesign actually is. Measuring the SUPR-Q prior to your redesign process will show you which areas of the user experience you can improve upon. Retesting with the SUPR-Q after you’ve made these adjustments then, will show you whether or not your users are actually experiencing the enhancements you think you and your team have made. 

Evaluating an e-commerce site 

Because of the trust and credibility section of the SUPR-Q, e-commerce sites might find this questionnaire particularly useful. More broadly, any website which has transactional capability on their website, might find the SUPR-Q as a useful tool for evaluating their website, if they haven’t before.  

Comparing your website to existing benchmarks and competitor sites

As mentioned previously, the SUPR-Q database allows you to benchmark your own website to the SUPR-Q scores of over 200 other websites. If you are interested in understanding how you stack up against the competition or against leading companies like Amazon, Walmart (etc.,), then the SUPR-Q might be of particular interest to you. 

How to get the most out of SUPR-Q 

• If your website isn’t intended for users to make a purchase or if they do not complete this portion of the customer experience on your site, then you might find that the trust/credibility questions of SUPR-Q do not apply to you. In this instance, consider subbing out the questions about 'purchasing' and 'conducting behavior' with the following: 
1. The information on the website is credible.
2. The information on the website is trustworthy.
• If you’re interested in the SUPR-Q, but feeling uneasy about the licensing fee, try out Qualaroo’s SUPR-Q template which is available on all subscription plans. While you won’t be able to compare your results to other websites, you will be able to compare your own website to future iterations of itself. Most importantly because it is calculated as a percentile rank score, you can still get a rough idea as to where you stand in comparison to other companies even if you can’t look at their exact results.  

Pros and Cons of SUPR-Q 

Pros

• Short, easy to understand and easy to implement
• The score itself is a benchmark because it represents a percentile rank. 
• Captures both user experience metrics and customer loyalty at the same time!

Cons

• Competitor data for other companies is shielded behind a steep licensing fee. 
• The SUPR-Q is most relevant and applicable to websites. Although it can be used in other contexts, your score will not be as insightful as a percentile rank since you would be comparing apples (an app for example) to oranges (website data). 
• On the plus side though, researchers are currently working on developing a normalized database for the SUPR-Qm, which evaluates SUPR-Q results for mobile applications. 
• It doesn’t identify bottlenecks or problems with navigation nor specific elements of the interface. Although results are still quantitative, they represent a subjective component of the user experience. 

Using SUPR-Q with Qualaroo 

Qualaroo Templates allow you to choose from two different versions of the SUPR-Q. You can use the full 8-item questionnaire or SUPR-Q without NPS if you are calculating this score in a separate campaign already. 

Set-up 

Start Running the System Usability Scale with Qualaroo in 3 simple steps! 

1. Select the right channel. 
• Decide where you want to run your survey (desktop, mobile web, prototype, or link) and select “Choose Template.”
2. Find your template. 
• Select the “SUPR-Q” template that works best for you.
• You can even type “SUPR-Q” into the search bar to find it.  
3. Create, edit and publish your survey! 
   • Make sure this is consistent across statements!
• The SUPR-Q questions are pre-filled and just about ready to go. 
• The only thing left to do is tailor your question to say “website” or “product”. 

Additional Tips 

Recommended targeting options

To get the most out of the SUPR-Q template, we recommend taking advantage of Qualaroo’s targeting options. Depending on the area of your company you wish to evaluate or the audience segment you are seeking to survey, targeting options may vary. However, these general recommendations can help you get your Nudge in front of the right users at the right time. 

• Something to keep in mind for timing is:
• Targeting based on exit-intent is a common strategy and is easy to do with Qualaroo. You may interpret a participant about to leave your page before completing a task as high perceived difficulty of the task or a lack of support opportunities available/easily findable. This could be an ideal time to launch the SUPR-Q. 
• Where should this survey appear?: 
  • Type in the URL (simple or advanced) where you expect the experience to naturally conclude.  
• For example, a confirmation purchase page (e-commerce)
• Who should be prompted to take this survey?: 
  • "Came directly to your site (typed in your URL, or bookmark)" +  "Only returning visitors". 
• Select 100% of all visitors who meet the conditions specified below
• When should this survey be displayed?: 
• Use Google Analytics as your guide to see what the average length of time people spend on a page is.
• If you have a specific page designated for the end of an experience as we mentioned above, you might also select “immediately after the page loads.”
• How often should this survey be displayed?: 
• Select “Continue showing until a visitor provides a response”

If you need help understanding all the options available within the Targeting Section, please visit this section of our Help Center.

Before and After you begin collecting responses 

Before

• If you’re planning to use an SUPR-Q questionnaire after usability testing, you may want to consider integrating Qualaroo with a session recording tool like fullstory or SessionCam. 
• If A/B testing your website is a use case that you’re interested in, Qualaroo offers an integration for Optimizely subscribers that helps further streamline this process. 
• Determine how long you want to run this campaign. Collecting responses continuously allows you to track improvements over time while setting an end-date can allow you to analyze data over fixed intervals. 

After

• Analyze your results to determine which patterns or trends are emerging.
• If need be, follow-up with outliers to gather more descriptive and qualitative feedback on their responses. This can be done by sending a follow-up survey or by reaching out to see if they have any interest in an interview. 
• If you have a central database where you are aggregating user or customer data, Qualaroo’s integrations (i.e. Segment, Tableau, Adobe Analytics and more) can help you funnel responses seamlessly for further analysis in context. 
• If you are on our Premium Plan, you can take advantage of our REST Reporting API to send Qualaroo response data to any endpoint of your choosing. 

Conclusion

The SUPR-Q is a short, yet in-depth tool that allows you to assess multiple aspects of the website experience. If this is a primary channel through which you interact with your users or customers, it’s a great starting point if you are looking to understand both usability and customer loyalty. Although you may not be able to compare your results to that of other websites, your score alone will give you a quick and fair reflection of where you rank in comparison to other websites by virtue of reflecting a percentile. 

If you have any additional questions about the SUPR-Q or how you can use this template, feel free to submit a request here. We are always looking for ways to ensure that Qualaroo can become easier for you to use, and more useful for your needs.