A Guide to User Testing: What It Is, How To Do It, and Why It’s Important

You’re an eCommerce brand manager ready to learn more about your customers and how they interact with your website. That’s great! 

It’s imperative, however, to understand that the journey you are about to embark on starts with user testing, and ends with an improved customer experience. The improvements you make could be the key to brand growth that you have been missing all along. 

Download Free: Usability Testing Guide

So, let’s dive right in. This article will cover: 

• What is user testing? 
• Three steps to conducting impactful user testing
• How well-run user testing can drive conversions (with examples)

What is user testing?

User testing is a pillar of customer research and conversion rate optimization, leveraged to reveal usability issues on your eCommerce site. Properly implemented, it can open your eyes to key pain points of your users and a host of quick wins to improve their experience. 

User testing can show you more than usability issues

While user testing is an excellent application of user research, you often already know general behavioral metrics like time on page, exit rates, etc., before you begin to test. The key is to expand your collection and include the context of quantitative and demographic data. 

By creating a more holistic picture of your customer’s journey, you will uncover usability issues and overall expectations that shoppers have for your brand. 

User testing vs. user research

While the terms user testing and user research are used interchangeably, user research is an umbrella term for various research methods. This includes interviews, surveys, focus groups, and more. User testing is just one branch of the user research tree.

Successful user testing can be highly effective in identifying customer pain points and sticking areas of your website. But, everything hinges on the way you screen and select users for tests. 
So, that’s how the steps to conducting impactful website usability testing start.

Three steps to conducting impactful user testing

Step 1: Be selective about your panel

Connecting with the right users for your test will offer not only better data but contextual clues and insights to indicate consumer motives, not just performance. 

To gather accurate test results and data, start by identifying your core audience demographics. Sift through your website’s data to collect sample data on age, gender, and location to understand on a base level who is converting and who is not. 

Use this demographic data as the first selection criteria for your panel. Most user testing services allow you to screen for gender, age, and location simply by checking a few boxes. 

Now, take things a step further to make sure you encompass more unique characteristics of your audience. To do so, you will need to craft some thoughtful screener questions. The answers to questions will help you identify psychographics that match your unique audience profile at a deeper level. 

For example, what is your approach to shopping for home decor?

• I go low stress. I prefer to make easy decisions from a few options to save myself time. [Reject]
• I take my time. I aim to find the best deal at the lowest price. [Reject]
• I take my time. I aim to find the best product, and I prioritize quality over cost. [Accept]
• None of the above. [Reject]

In order to convert more website visitors into buyers for clients, start with a clean slate and target consumers who have not already learned the nuances of a site. This way, the user isn’t naturally overlooking pain points that they learned to navigate in previous visits. 

To do this, screen for potential user testers who have heard of your in-market competitors but don’t know your site or brand. 

Here’s an example: 

Which of the following brands have you purchased from?

• Competitor 1 [Accept]
• Competitor 2 [Accept]
• Your site [Reject]
• Competitor 3 [Accept]
• Fake company [Reject]
• None of the above [Reject]

After you screen for the most critical criteria, you also want to consider how users shop and their intent to purchase. Avoid users who have no intention to purchase, and consider screening for panelists who shop online at least once a week. 

In summary, as you select your user testing panel: 

• Screen for your unique audience demographics
• Layer on psychographic questions that align with your ideal user profile
• Avoid users that know too much about your site/brand
• Talk to users with an intent to purchase

Now, you are on your way to a successful user test, but this is where the real work begins.

Step 2: Ask the right questions to surface meaningful feedback

The thoughts and responses you get from user testing make the efforts worthwhile. So how do you make sure you are surfacing meaningful feedback? It’s all about the questions you ask and what you task your carefully selected panel with. 

While specifics will change based on your brand and goals, below are a few guidelines to help navigate the creation of your user testing questions. 

1. Avoid biasing your audience by forming an intro question that does not prime users with the purpose of the site. Here is an example: Imagine you are browsing for home decor online. Without leaving the homepage, describe in your own words what you think you can do on this site? What would you come here for? 

2. Outline key pain points from data you already analyzed to help structure your questions and tasks. This will reveal why users behave the way they do on your site. For example, if you notice from the data that users are most often abandoning the product page, you may want to focus some of your questions on what content is working there and what isn’t. For example, what, if anything, is missing from the product page that you would need to know in order to complete your purchase?

3. The first few minutes of a user testing session are typically mimicking the most ‘natural’ interaction someone has with your site. A frequent mistake is being too narrow-minded or specific with tests and losing the user’s natural behavior. To avoid this, use a time limit and after a few priming questions, start testing with “Explore this website in whatever area interests you. Spend no more than a few minutes on this task.” This produces close-to-natural behavior that will show you what’s grabbing user attention and how they would organically behave on-site before moving to the next task. 
4. Before launching user testing, always run a pilot (a small-scale, preliminary test) to uncover unclear questions and areas for improvement in your test. 
5. Don’t run too many tests. A study has found that roughly five users per usability test uncover about 80% of usability issues.

Review these guidelines with each user test you conduct. Even trained researchers review the basics continually during their work and studies. Learning to remove bias, set learning goals, and avoid unnatural behavior will build higher impact user tests.

Launching a pilot test, and moving in small, iterative testing cycles means you’re never taking risks by investing precious resources into a potentially faulty test. 

Now is the most exciting part of the user testing process: receiving and analyzing results to create winning strategies for your brand. 

Step 3: Analyze and leverage your data for winning strategies

Though exciting, analyzing and leveraging the data you collect from user tests can be an intimidating item on your to-do list. 

Your tests have likely pointed out a host of usability issues and maybe even some customer experience pain points. Below you’ll learn how to sift through data, prioritize issues, and come up with winning A/B tests and strategies to improve your site.

Download Free: Usability Testing Guide

Analyze the results

As a starting point for analyzing the results of your test, always return to your objectives. Remind yourself of the goals you had for the tests, and pick out insights and feedback that will get you closer to those goals. 

Then, while you sift through the results of your user tests, remember that the suggestions made should not be taken literally. While feedback from consumers is valuable, it does not often address the root of the issue. Gather the insights efficiently so you can come up with a solution that meets the needs of the customer and the business, not the self-identified frustration or problem. 

Another technique to better understand the sentiment of users is to pull quotes from the testing sessions. This can help identify areas of misunderstanding, i.e. when the user says they are going to do one thing but gets lost along the way. Direct quotes also offer insight into user expectations that are not being met.

Always remember as you go through results to take the user ratings with a grain of salt. For example, some users will rate a website as highly navigable even though they took much more time than ideal for reaching the desired result. This could be because they think they failed at navigating the site when really the site failed them. To do this, pay attention to how long tasks take. 

Identify improvement areas

Once you have analyzed your user testing results, it’s crucial to prioritize the issues to be addressed. You first want to solve the most common or frequently encountered problems and then move to minor or specific glitches. 

One way to do this is to create a spreadsheet of all the identified problems. You can organize the document based on the impact the issue had on user experience and the frequency at which it occurred. That sheet becomes a prioritized checklist for optimization. 

Check out our insightful conversation with Kevin Anderson to know how you can use this data and build a culture of experimentation within your company.

Run A/B tests with VWO

With a prioritized checklist in place, it’s time to create solid hypotheses around the identified problems and start running experiments. VWO’s A/B testing tool enables you to optimize key business metrics by figuring out visitor behavior, test ideas, and engagement across the users’ entire journey.

Ubisoft increased its lead generation by 12% utilizing A/B testing. Their team collected visitor data using heatmaps, on-page surveys, scrollmaps and analyzed data to build a hypothesis for an experiment to simplify the buying process and increase conversion on the Buy Now page, which was their key performance indicator. Below is a screenshot of the winning variation:

The layout of the entire page was redesigned and a test was run for 2 months. The redesigned version increased the lead generation rate from 38% to 50%. 

Start your free trial on VWO and create an iterative and data-driven process for consistent improvements in your website conversion goals. 

Well-run user testing can drive conversions

Now you know the three steps to conducting impactful user testing. Let’s take a look at the results that can come from your efforts. 

At the end of the day, the reason you are gathering this data is to improve your site and, in turn, drive more conversions. 

Take the DTC office furniture brand, Fully, as an example. As the world transitioned to work from home in 2020, the brand had longer lead times on key products like their Jarvis standing desk. While this could have been an out-of-stock issue, user testing and creative thinking turned it into an opportunity to highlight other equally well-made products. 

Fully conducted remote usability testing, using a similar 3 step process as outlined above. Heatmap analysis showed high engagement with the filters and indicated that users who were not already familiar with Fully might not know what office furniture best suits their needs.

As a solution, improved product filters and categories were tested to increase visibility for the lesser-known product lines and provided users with a focused approach to navigating the entire catalog. This one test, which started with user testing, resulted in a 75:1 ROI for the brand.

Another example of powerful user testing driving conversions comes from Beckett Simonon, a hand-crafted leather shoe and accessory brand. 

The brand was looking to ensure paid acquisition efforts were converting effectively. They needed a website optimization solution that would result in sustainable, long-term improvements on their conversion rate. 

With remote user-testing sessions, it was revealed that users relied heavily on product images to make purchasing decisions. It also became apparent that users did not understand the differentiator of the brand. 

To solve this, the integration of value-based messaging in key image-driven moments of the customer journey was tested. Below is the original site and the variant tested based on insights from user testing. 

This result not only better demonstrated the core values of the brand but also resulted in a predicted annualized ROI of 237%. 

User testing unlocks insights that can drive results like Fully’s and Beckett Simonon’s for your eCommerce brand.

Get started with user testing

So, you’re ready to get started with user testing. Beyond usability, you now know that user testing is about both the action and understanding the thought behind each action. 

As a jumping-off point for your user testing, try out a tool that allows you to filter user-testers by age, income, gender, country, internet expertise, and type of device used to access your site. There are plenty of well-established options available. 

Remember, user testing is not just a pre-launch strategy for eCommerce brands. Ongoing testing is the best way to keep your customer journey enjoyable and the path to purchase smooth. That is the backbone of conversion rate optimization: constantly optimizing your website to keep customers happy, purchasing, and coming back.