Guide to user testing – types & timeline

In fact, pairing user testing alongside market and product fit is the trifecta in building your product strategy. When teams do these things, they are more efficient in every other area of development, accelerating product roadmaps and finding ROI faster.

This knowledge only comes from solving your user’s underlying problems, which comes from testing.

In this guide to user testing, we’ll take you through the types of user testing, when each should be used, and how to use them in a realistic product life cycle.

Types of User Tests

There are two categories of user testing, qualitative vs quantitative.

Qualitative testing is learning about a user’s interpretations and feelings, while quantitative is number-based and measurable. An example might be interviewing a single user about their needs and motivations vs A/B testing a single interface element or user flow.

When conducting your quantitative data, be sure to account for statistical significance, otherwise, your results could be by chance and not a real consensus of users.

For the best results, you need both qualitative and quantitative testing. Here are the top user tests and how they help.

Usability testing

One of the most important and popular tests, usability testing asks a user to complete a particular task with feedback while a moderator watches and documents. These kinds of tests are essential because they put a real human behind a device.

With this, you can learn about their problems, how they navigate, how they search for information, and what they like/dislike about a certain product.

Direct observation

With direct observation, you observe without interaction. Seeing a user easily find information or struggle to complete a sign-up can shed light on how easy your application is to use.

Prototyping

With prototyping, your user is reviewing a prototype, not a full-fledged app. Sometimes these prototypes are low fidelity wireframes, sometimes they’re high fidelity and clickable. Since development is expensive and time-consuming, you don’t want to begin developing without testing some kind of prototype.

A/B testing

Comparing two options to find the superior one can be a fast way to iterate. Is a user more likely to respond to a “Contact Us” CTA or “Let’s Talk”? Does a blue button work better than a red button? With A/B testing, you can get useful data and make rapid-fire decisions.

Treejack testing

Treejack testing is ideal for information architecture testing, which is essential for any complex website or product. You can ask questions like, “if you wanted to find the location nearest to you, what page would you click on?” and then record all interactions.

Eye-tracking and heat mapping

Eye-tracking and heat map testing can be expensive, but it’s worthwhile for some products. With these tests, you can measure where your users are looking and where they’re clicking on your app– to understand if you are providing the right information in the right place.

In-person vs remote

You can conduct all of these tests in-person, remote, or automatically. Remember to take these different environments into account when analyzing data, because different atmospheres can affect users’ opinions and how they interact with technology.

When & how often should a team test?

Like most answers to design-related questions, “It depends.”

Minimum viable products (MVPs) need different testing compared to a legacy app. A single-page web app needs different testing versus a large e-commerce store. So, unfortunately, there is no one size fits all solution.

A robust discovery session is a good practice. This process helps strategists understand the problem space and craft a program to tackle the issue. It also helps designers determine how to test and when.

However, we can provide some general guidelines which are:

Test often

A strong design process starts with discovery and definition, before diving into ideation. Then, repetition is essential.

Ideate – Prototype – Test as needed

Ideate – Prototype – Test as needed

Every time you iterate, you need to ask yourself, “should we test this?” While a single colour swap won’t require standalone testing, a new feature will.

You also have a budget to consider. Testing every small thing will cost you time and money, so have discussions, ask questions, and keep testing on your team’s mind.

Lightweight iterations = quantitative tests

There are always exceptions to this rule, but it’s generally true.

A “lightweight iteration” is difficult to define, but it’s likely something that can be A/B tested and decided upon with analytics.

Heavyweight iterations = qualitative tests

Also exceptions here, but when you do more intensive iterations, like developing new features or launching to a new audience, you need to sit down and talk to users. Understanding not just what they like, but why they like it can be incredibly insightful.

Internal vs external testing

If you’re in the very beginning stages of product development (think wireframes), we recommend testing with internal stakeholders.

Don’t go outside of your organisation until you have high-fidelity prototypes. After all, every product needs stakeholders to sign off before digging into the design.

(About) PRIME™ Creative

PRIME™ Creative is a brand strategy and digital design agency that brings brands and culture together. With a belief that culture drives commerce, PRIME leverages shared values and ideals to inform strategy and design, creating experiences that inspire life and inspire action. With a growing client list, PRIME specializes in insights, strategy-driven and design-led company development, and digital innovation / user experience.