System Usability Scale (SUS)

System Usability Scale Template

What you’ll find in this article:

• What is the System Usability Scale (SUS)?
• How does SUS work?
• When to Use SUS 
• Best practices for SUS 
• Pros and Cons of SUS 
• Using SUS with Qualaroo
• Conclusion

What is the System Usability Scale (SUS)?

The System Usability Scale (SUS) measures the perceived usability of a system, or as we call it nowadays: a website, app, or other digital product. SUS is actually one of the most popular standardized usability questionnaires. Although it is considered a unidimensional test for usability, the questionnaire actually covers both usability and learnability in its scope. In particular, it is most recommended for software applications although it’s considered versatile enough to be used on anything with a user interface. 

SUS was developed in 1986 by John Brooke while he was an employee of Digital Equipment Corporation, (now part of Hewlett-Packard). Brooke and others often refer to SUS as the “quick and dirty” usability scale because of how easy it is to implement. However, the reliability of SUS results has led many UX professionals to call out that perhaps SUS isn’t so “dirty” after all. 

How does SUS work?

The System Usability Scale Questionnaire consists of 10 general statements on a 5-point Likert scale. The scale ranges from “Strongly Disagree” - 1 to “Strongly agree” - 5. 

However, one unique aspect of the SUS Questionnaire is that these 10 items are designed to have an alternating tone on every other statement. Here’s a copy of the questionnaire so you can see check out exactly what we mean. 

The Questionnaire

1. I think that I would like to use this [website/ product/ tool/ software/ prototype] frequently
2. I found the [website/ product/ tool/ software/ prototype] unnecessarily complex
3. I thought the [website/ product/ tool/ software/ prototype] was easy to use
4. I think I would need the support of a technical person to be able to use this [website/ product/ tool/ software/ prototype].
5. I found the various functions in this [website/ product/ tool/ software/ prototype] were well integrated.
6. I thought this [website/ product/ tool/ software/ prototype] was too inconsistent.
7. I would imagine that most people would learn to use this [website/ product/ tool/ software/ prototype] very quickly.
8. I found the [website/ product/ tool/ software/ prototype] very cumbersome to use.
9. I felt very confident using the [website/ product/ tool/ software/ prototype].
10. I needed to learn a lot of things before I could get going with this [website/ product/ tool/ software/ prototype].

As you may have noticed, the tone of these 10 statements alters on every other item. Odd statements are phrased in a way that is more favorable or positive. Meanwhile, even-numbered statements are worded to have a more negative or unfavorable tone. 

This alternating presentation is intended to help account for extreme response bias, (e.g: someone answering all 1s or all 5s throughout). 

Score Calculation 

Step 1: For odd-numbered items, subtract 1 from the user response. 

• User response - 1 = new value

Step 2: For even-numbered items, subtract the user response from 5. 

• 5 - user response = new value 

Why these subtractions?

As mentioned earlier, since odd items offer positive statements while even-numbered items present negative statements, the raw user responses aren’t comparable to one another. 

Doing these simple subtractions puts all values on a scale of 0-4 and makes 4 the most positive response regardless of how the original statement was phrased. 

Source: Medium 

Step 3: Sum up the newly calculated values for a total score that will range from 0 to 40. 

Step 4: Multiply this sum by a factor of 2.5

• Doing so will give you a final SUS score between 0 to 100 which can be more quickly understood than values on a 40-point scale. 

Step 5: Repeat steps 1-4 for each respondent. 

Step 6: Average your participants’ individual scores to get a final SUS score.

*Tip: Doing this in Excel or Google Sheets can help you save time on scoring. Use this template for SUS scoring or consider downloading this free Google Sheets extension which includes reliability testing. 

What Your Score Means

Even though scores are put on a scale of 0-100, they do not represent percentiles. In other words, an SUS score of 50 would not be considered average or at the 50th percentile. The actual average score for SUS is 68 across systems.

You can use the graph below from MeasuringU to help figure out where your SUS score falls within percentile ranks. 

Source: measuringu.com

When to use SUS

The System Usability Scale is most commonly used as a qualitative follow-up to usability testing rather than for diagnostic use. However this can look entirely different, depending on the needs of your organization. 

Using SUS after a usability test 

When it comes to administering the SUS after a usability test, you have a few options. You could share it at the end of the test to assess your overall system or product for a broad overview of perceived usability. On the other hand, you could also do focused functionality testing and assess specific portions or features of your product. This is especially useful if you’re looking to target and understand problem areas or if you’re trying to gauge user reactions to new feature rollouts or product launches. 

For Prototypes and Live Products

The System Usability Scale can be effective during both prototyping and product validation. 

With prototypes, you can catch usability issues in your product early on before you’ve spent development resources building it. Once you’ve made changes, you can also retest the prototype to see whether or not your SUS has improved. If you are interested in A/B testing two different designs or versions of your website, you can use SUS to decide which direction you and your team will go. 

Keep in mind if you use SUS during prototyping that your score shouldn’t really be compared to existing benchmark or percentile data that has been averaged based on live products. However, you can create benchmarks within your company and compare prototypes to those that are at the same stage of development. This can be especially useful as a form of product testing if your company develops multiple products.  

Of course, once you’ve actually launched your final interface or product, SUS can help measure users’ attitudes on a more regular basis. You can do this as a stand-alone pulse survey that only happens after a user has finished a specific task or schedule it at regular intervals, such as quarterly or bi-annually. Even if you haven’t made any recent changes to the interface, user’s perceived usability can still fluctuate since new trends and standards in the market are constantly evolving and introducing new habits that impact how people use technology. 

Best practices for SUS 

• Best administered electronically, especially for scoring purposes. 
• Combine your results with task testing or SEQ testing and note where issues arise in the user experience to quickly pinpoint which areas of your interface are most impacting SUS scores. 
• When choosing which word you want to use in your statements (website/ product/ tool/ software/ prototype), choose the word that most closely fits how your users would refer to the system. 
• More importantly, be sure to use this same word in all 10 statements to prevent any possible user confusion and for the sake of consistency.

Pros and Cons of SUS

Pros

• When combining SUS with usability testing, you actually don’t need a very big sample size in order to get relatively stable and reliable results
  • Five is actually considered a magic number in early-usability tests. (Source).
• SUS is seen as an industry standard and has been referenced in more than 1300 different articles and publications
• SUS produces reliable and repeatable results. Although some critics might argue that the System Usability Scale has a certain amount of redundancy to it, others would argue that asking users to evaluate these statements in nuanced ways is what makes SUS reliable and allows for stronger insights. 
• SUS is device agnostic and applicable to anything that has an interface. (For example, the creator, John Brooke used it for green screen terminals– think old school green text mainframe computers.) 

Cons

• The alternating tone does have its side-effects. 
• Sometimes participants may not notice the tone of the statement is alternating. This might cause them to select a lower number on both odd and even numbers if they disagree with only one. 
  
  • Be sure to leave your scale labelled. Luckily, using Templates with Qualaroo already does this for you!
  • Double check any scores that stand out as red flags and follow up if necessary. For example, if a respondent answers all 5s on the questionnaire, that’s very likely due to user error rather than indicative of user sentiment
• It’s easy to incorrectly score the questionnaire if you don’t account for even items being inversely scored
  • Be sure to code items correctly when scoring
• Long-time users can skew your data. 
• The more familiar a user is with a system or application, the likelier they are to rate it favorably on SUS. Because repeat users have more experience with the product or application, they will probably agree that it’s easy to use and wasn’t hard to learn. Newer users on the other hand are much closer to their initial onboarding experience and will have a better account of learnability. 
• When targeting users consider showing the SUS questionnaire to a healthy mix of new or first-time users, long-term users and everything in between 
• Calculating the score isn’t as plug and chug as averaging the data set. You actually have to make sure you are converting your raw results into usable data. 
• It doesn’t measure specific characteristics like usefulness or accessibility. It also doesn’t help identify navigation issues, bottlenecks or problems with particular elements of the interface.

Using SUS with Qualaroo

Set-up 

Start Running the System Usability Scale with Qualaroo in 3 simple steps! 

1. Select the right channel. 
• Decide where you want to run your survey (desktop, mobile web, prototype, mobile app or link) and select “Choose Template.”
• Don’t worry–you can run this survey on as many channels as your organization needs by creating additional surveys so that you cover the different domains your organization spans.
2. Find your template. 
• Select “System Usability Score - SUS'' from our list of pre-vetted templates. 
• You can even type “SUS” into the search bar to find it or filter by “Validate” templates.
3. Create, edit and publish your survey! 
• All 10 statements of the SUS Questionnaire have already been entered for you. 
• The only thing left to do is select the right word choice to describe your system: website/product/software, etc., 

Additional Tips 

Getting the most out of the SUS Questionnaire Template

To get the most out of the SUS template, we recommend taking advantage of Qualaroo’s targeting options. Depending on the area of your company you wish to evaluate or the audience segment you are seeking to survey, targeting options may vary. However, these general recommendations can help you get your Nudge in front of the right users at the right time. 

If you have a more specific use case, feel free to get in touch with your designated customer success manager to discuss further options that you can take advantage of. 

Recommended targeting options

• Where should this survey appear?: 
• Only where active users will see it 
  • This might be behind the login wall after a user has completed a specific task or after the individual has gone through a customer support experience.
  • Type in the URL (simple or advanced) where you expect the experience to naturally conclude.  
• Who should be prompted to take this survey?: 
  • Select 100% of all visitors who meet the conditions specified below
    
    • "Came directly to your site (typed in your URL, or bookmark)" +  "Only returning visitors". 
• When should this survey be displayed?: 
• After the log-in wall to visitors who are returning
  • However, SUS is not suitable for displaying to first-time users as it can negatively impact their onboarding experience.
    
• You can also schedule a time delay that aligns best with your user behavior. Use Google Analytics as your guide to see what the average length of time people spend on a page is.
• If you have pages set up for the end of a certain path like thank you (for purchasing, downloading, subscribing) screens or success pages designated for a specific experience, you might also select “immediately after the page loads.”
• How often should this survey be displayed?: 
• Select “Continue showing until a visitor provides a response”

If you need help understanding all the options available within the Targeting Section, please visit this section of our Help Center.

Before and After you begin collecting responses

Before

• Determine how long you want to run this campaign. Collecting responses continuously allows you to track improvements over time while setting an end-date can allow you to analyze data over fixed intervals. 
• If you’re planning to use an SUS questionnaire after usability testing, you may want to consider integrating Qualaroo with a session recording tool like Fullstory or SessionCam. These tools can help you track quantitative metrics, such as error rates and task timings which can help you further interpret the results of your SUS data. 
• If A/B testing is a use case that you’re interested in, Qualaroo offers an integration for Optimizely subscribers that helps further streamline this process. 

After

• Analyze your results to determine which patterns or trends are emerging.
• If need be, follow-up with outliers to gather more descriptive and qualitative feedback on their responses. This can be done by sending a follow-up survey or by reaching out to see if they have any interest in an interview. 
• If you have a central database where you are aggregating user or customer data, Qualaroo’s integrations (i.e. Segment, Tableau, Adobe Analytics and more) can help you funnel responses seamlessly for further analysis in context. 
• If you are on our Premium Plan, you can take advantage of our REST Reporting API to send Qualaroo response data to any endpoint of your choosing. 

Conclusion 

The SUS Questionnaire is one of the most reliable and commonly used questionnaires after usability testing. It’s a well-researched, well-validated thorough tool for evaluating your product’s usability and learnability. While there are many other ways to assess usability, this survey is cost-efficient, straightforward, and will help you standardize the way your company examines best practices in usability. 

If you have any additional questions or would like to get in touch with a customer success manager about setting up this template, please submit a request here. We are always looking for ways to ensure that Qualaroo can become more usable and quickly-learnable so that you can accomplish your goals!