Should you hire an agency for your A/B testing or should you do it yourself?

To be the most successful with your website testing efforts and results, you need more than just a testing tool like VWO!
You need a team of experts to run your testing program – both the strategy behind it and actually setting up and running tests. But is it easier to build this team in-house or outsource to a website testing agency? What is the best option? This is a very common question from online businesses, so here is a quick guide to help you decide what options are best for you, including some questions to ask potential testing agencies.

Download Free: Conversion Rate Optimization Guide

Outsourcing Your Testing – Pros and Cons

CRO Methodology by Conversion Rate Experts

Your first option is to outsource all your testing efforts to an agency that specializes in website testing and conversion rate optimization. They handle everything for you, from creating test strategies and ideas, to test design, implementation and analysis (these are known as full service testing agencies). Here are the pros and cons of this method:

• A great option if you are new to website testing and don’t have many internal resources needed to help run an effective testing program.
• Usually a much quicker way of getting better results, as the agency provides all necessary testing roles and expertise (for example test managers, test strategists, test developers and designers, project managers).
• Often much more expensive to get running initially (often requires a large upfront budget, unless you use a pay for performance agency).
• Agencies won’t understand the more complex and intricate details of your business and goals as well as you do, and may lack detailed experience in your industry.
• You have less control over the testing (agency takes the lead and decides on what is best).
• Testing agencies can help break down internal testing barriers, and help build understanding of the need to test and the great impact of it.
• You don’t have to learn A/B testing tools and best practices if you don’t want to – you can leave everything up to the experts at the agency.
• Allows you to learn best practices and gain experience from experts within the agency you chose, allowing you to eventually use these to build a better in-house testing team.

Ideal for: Outsourcing is the best option for businesses with large budgets for testing and want to get results as quick as possible.

Examples of full service testing agencies: Adobe Digital, Wider Funnel, Ion Interactive, Conversion Rate Experts, Conversion Factory.

Testing In-house – Pros and Cons

Your second alternative is try and run your website testing efforts internally (in-house), by creating your own expert team that handles all aspects of testing. Here are the pros and cons of choosing this method:

• Takes a long time to build up a good team and expertise (you need to find, hire and train for numerous roles like testing manager, testing lead and test developers and designers).
• Often takes much longer to start achieving great test results (longer learning curve for knowing best practices and advanced test ideas).
• Lower initial startup costs to begin testing than using a testing agency (unless you use a pay for performance agency).
• More control over the testing process than if you use an agency – you choose the strategy, what gets tested and when.
• You know the detailed intricacies of your business the best, better than an agency would, so you can create advanced business goal tests often better than they could.
• Ultimately though, companies that are most successful with their website testing programs usually eventually bring testing back in-house.

Ideal for: Testing in-house is best suited for online businesses who prefer to learn and be more self-sufficient with their testing efforts, especially those with small testing budget, or very complex online business needs.

Hybrid In-house/outsourced Testing

VWO’s in-built heatmap functionality helps visitor behavior analysis

Another option you can pursue is a hybrid of the above options – to only outsource your testing strategy but keep test implementation and design in-house. This approach combines benefits of both, for example it usually costs less than outsourcing your test development and design. Most of the full-service agencies mentioned above can help with just strategy, and there are several consulting companies that specialize in just offering testing strategy advice.

Examples of strategy focused consultancies:  SiteTuners and Rich Page: Website Optimizer.

Ideal for: Online businesses who just need a kickstart for creating test ideas or only need help with strategy, and can handle the creation and implementation of tests in-house.

Download Free: Conversion Rate Optimization Guide

Questions You Should Ask Potential Outsourcing Agencies

To help you decide if an outsourced testing agency or consulting firm is right for you, I suggest you reach out to many of them listed above and ask them some questions like the following:

• How hands-on is the testing agency? How much do they collaborate with your team?
• How much do they charge, and what is their pricing model? (some charge per test, depending on traffic level, or performance based)
• What service levels do they offer? Full service/strategy only?
• What experience and clients do they have in your industry? (this will help them hit the ground running if they do.

If you decide to choose the testing outsourcing path, you should always start with a proof-of-concept contract with the testing agency to make sure they deliver the results you expect.

Final Thoughts

Overall, if you can obtain a good large budget for it, I would start off with outsourcing your testing efforts to a full service agency or strategic consultancy, and then gradually learn from them and build-up your own internal testing team over time.  Indeed, many businesses struggle to test effectively in-house because it takes so long to build an expert team, unless they have a high level of buy-in and resources for testing.

There is no right website testing solution for every company though – it will ultimately depend on what internal testing resources or expertise you currently have available, how much budget you have, and even how much testing buy-in from senior executives you have.

Now over to you – what testing method have you found works best for your online business? Success stories (and even horror stories) are welcome!