Adobe Creative Cloud

Design

February 23, 2017 /UX/UI Design /

The Top 5 User Testing Methods

In an industry devoted to the people who use our products, services, and apps, user testing is paramount. The main goal of user testing is to inform the design process from the perspective of the end user. User-centered design is focused on designing for real users, and user testing tells us who that person is, in what context they’ll use a product, and what goal they are looking to achieve.

UX researchers have developed many techniques over the years for testing and validating their ideas, ranging from well-known lab-based usability studies to those that have been more recently developed, such as unmoderated online UX assessments and guerilla testing.

Some of the most popular forms of testing are usability testing, focus groups, beta testing, A/B testing, and surveys:

1. Usability Testing

Usability testing is the process of watching/tracking an actual user while they use your product to see if it’s in fact usable. Usability testing is the best way to understand how real users experience your website or app. It’s also flexible for collecting a range of information about users, and it’s easy to combine with other techniques. Usability testing is a cornerstone of UX practice.

When it comes to usability testing, one of the most important decisions you’ll make is whether someone should moderate the session.

Moderated Usability Testing

Moderated usability testing is practiced by professionals looking to obtain feedback from live users. During a moderated test, moderators are ‘live’ with test participants (either in person or remotely), facilitating them through tasks, answering their questions, and replying to their feedback in real time. Live communication with test participants is a strength of this type of testing, because nothing beats watching participants in real time, and being able to ask probing questions about what they are doing.

Tip: If you want to use moderated testing, make sure you follow these 20 tips for moderated usability testing.

Moderated usability testing is usually held inside usability labs. Credits: usabilitygeek

When To Use

Moderated tests are recommended during the design phase – when a team has a design that hasn’t yet been fully developed. You can run a moderated test to find the potential issues of your working prototype. By watching participants reactions on your prototype, you can gather baseline data that can save you from spending a lot of design and development time on a product that are difficult to use.

Things To Remember

A moderator can help probe the participant to delve deeper, keep the participant on track, and help clarify any confusion. However, a very common mistake moderators make is to tell a participant what to do. There is a fine line between guiding and helping a user. Thus, you need to find a balance to keep the participant on task, while not messing with their natural experience. When this balance is struck properly, even the most convoluted of tasks can provide rewarding feedback.

Unmoderated Remote Usability Testing (URUT)

Unmoderated remote usability testing, as the name implies, occurs remotely without a moderator. It offers quick, robust, and inexpensive user testing results.

Unmoderated tests can be done virtually anywhere at any time, by anyone who meets your criteria. Credits: UserZoom

This method is usually based on the use of usability testing tools that automatically gather participant feedback and record their behavior.

URUT tools conduct usability testing by asking participants to complete a series of tasks using your product and answering questions about their experience.

URUT has following benefits:

  • Participants complete tasks in their own environment without a moderator present, which leads to the product being used naturally.
  • URUT is conducted online much like a survey with pre-determined tasks, so it can be completed in the participant’s own time without requiring the hassle of coordinating schedules.
  • Unmoderated tests can also be run concurrently, allowing for a much greater volume. The turn-around time for unmoderated tests is often significantly faster than that of moderated tests. Data can be collected in as a little as a few hours depending on sample size and testing criteria.
  • The cost is usually quite low since you don’t need to pay for moderators or an equipment setup. You can get maximum value for minimum cost when tasks are written as clearly as possible.

When To Use

  • When you need to obtain a large sample in order to prove key findings from your initial moderated research.
  • When you have very specific questions about how people use a user interface for relatively simple and straightforward tasks.

Things To Remember

  • URUT should not be used as a replacement for moderated usability testing. Instead, it’s best when you use it in conjunction with moderated testing.
  • The lack of a moderator means less control, less personal observation, and a higher risk of confusion. Thus, to run a test successfully you need to set clear expectations for participants—it’s crucial to ensure that tasks are clear and user-friendly.
  • Be mindful of how much time participants spend with test. Kyle Soucy suggests an unmoderated test should be 15–30 minutes in duration, and comprised of approximately 3–5 tasks, because the dropout rate tends to increase if a test takes longer.

2. Focus Groups

Focus groups are a tried and true method of communication between a researcher and users. In a focus group, you bring together 6 to 12 users to discuss issues and concerns about the features of a UI. The group typically lasts about 2 hours and is run by a moderator who maintains the group’s focus.

Tip: Check out the article The Use and Misuse of Focus Groups on how to effectively utilize focus groups for user testing.

Focus groups provide a top-of-mind view of what people think about a product.

When To Use

Focus groups can be a powerful tool in system development. This technique can help you assess user needs and feelings both before a product’s design and long after its release. In website or mobile app development, the proper role of focus groups isn’t to assess design usability, but to discover what users want from a product—their personal thoughts and preferences.

Things to remember:

  • Focus groups shouldn’t be used as your only source of user testing data. They are a rather poor method for evaluating interface usability as individuals rarely get the chance to explore the product on their own.
  • It’s recommended to run more than one focus group, because the outcome of any single session may not be representative.

3. Beta Testing

Beta testing allows you to roll out a near-complete product to individuals who are happy to try it and provide critical feedback. This testing method allows you to ask users questions after they have the new product, track their usage and have them file bug reports.

When To Use

You should use this testing when your product is near complete and you want to put it in the hands of the end users to gather feedback. Beta testing a good way to market your product and get constructive feedback in order to refine the design to improve the product.

Things To Remember

Sufficient testing should be carried out before releasing a product to the customers. Naturally, you don’t want your users to find and report bugs, you simply want their feedback on product features and usability.

4. A/B Testing

An A/B test is ideal as the appropriate testing method when designers are struggling to choose between two competing elements. This testing method consists of showing one of two versions randomly to an equal number of users, and then reviewing analytics on which version accomplished a specific goal more effectively.

Using A/B testing you have the opportunity to study the behavior of users, how they act in different scenarios.

Tip: You can define stronger A/B test variations through UX research.

When To Use

A/B testing is good when trying to detect small differences in designs. This testing is particularly valuable when comparing a revised screen to an older version.

Things To Remember

With A/B testing you only find the best option from among the available variations. These variations should be selected very carefully. If the variations are only based on internal experience and opinion, the testing won’t find the optimal design.

5. Surveys

Questionnaires and surveys are an easy way to gather a large amount of information about users, with minimal time invested. A researcher can create a survey using tools like Wufoo, SurveyMonkey or Google Forms, send it out, and receive hundreds of responses in just minutes. The right questions can uncover customer needs, desires, and pains.

Tip: Here is a great step-by-step guide of creating a survey.

When To Use

Surveys can help you accumulate quantitative data about overall user satisfaction or collect quantitative data to support qualitative research findings.

Hundreds of responses can be seen at once. Credits: SurveyMonkey

Surveys are also good when you need to gather feedback about a brand new feature.

Things To Remember

  • You can’t study user behavior with surveys. If you want to study how your visitors behave or what usability problems they face during an interaction with your product, consider other research methods.
  • Creating a survey looks like a quick and easy task, but in reality it is the opposite. A significant amount of time should be dedicated to preparing surveys. It’s important to get the questions right and direct them at the right audience.

Conclusion

User testing is an essential part of the design process – it’s a fantastic way to understand how your user base interacts with your product. As you just saw, different types of user testing suit different types of goals. Ultimately, the best format of user testing depends entirely on what your product is, what you’re looking to learn about it, and how much time you have available. So it’s up to you to consider which method will best suit your needs in order to gather the most valuable feedback on the user experience of your product.

Save

Save

Save

UX/UI Design

Design. Prototype. Share. All in Adobe XD.

Go from idea to prototype faster with Adobe Experience Design CC (Beta), the first all-in-one tool for creating and sharing website and mobile app designs. Test drive XD and tell us what you think.

DOWNLOAD