Success Story

Warner Music Group Increased Its A/B Testing Velocity Using VWO

5% Conversion

About WMG

VWO spoke to Brandon Gentry, Director of User Experience at Warner Music Group (WMG), and his team, which is responsible for managing websites of the artists that WMG represents. Over the course of 18 months, the UX team at WMG developed a robust framework for deploying websites that are fine-tuned for conversions and user acquisition.

The UX team at WMG ensured that the website optimization process delivers maximum results without any compromise to the websites’ user experience. Along with optimization, it was also important that the websites adhered to the artist’s essence. This success story highlights the conversion optimization challenges faced by the UX team at WMG, and how it has overcome them.

In my role at Warner Music Group, I aim to promote a culture of data-driven creative design. We design, build and maintain hundreds of artists’ websites. VWO’s suite of tools help me quickly and easily find conversion optimization wins that I can standardize across our whole enterprise. I don’t like to use the phrase “best practice” unless I can prove it, and my proof is in the data provided by VWO.

Brandon Gentry
Director, User Experience

Observations

Before using VWO, WMG’s UX team worked with a third-party website optimization agency. The third-party partner provided them invaluable data and validated conversion optimization as a strategy to improve user experience and acquisition. However, Brandon realized that their efforts had a better scope if the whole process was carried out in-house with minimal assistance. These were due to the following reasons:

Approval-dependent process: The third-party partnership led to a slow turnover of results. WMG’s UX team was responsible to provide approvals on development assets, testing hypothesis, and design assets, which made the process slow.
Budget constraints: Due to the nature of the partnership and the budget allocation, the number of tests carried per year were limited.
Slow testing speed: Web testing required time. Getting label approval and webmaster approval made the entire process slow. Over the course of a year, WMG could only complete 10 tests.

WMG’s Conversion Optimization Goals

In April 2016, WMG built a dedicated in-house testing team and signed up with VWO to ramp up its conversion efforts. The complete process was streamlined by stitching together the design, user experience, and production teams in a close environment.

Previously, WMG’s UX team was able to run 10 tests over the course of a year with its agency partner. Now, they were looking to improve these numbers by at least 2X with overall improvement in the conversion rates as well.

The second goal for the team was to measure the scope of a given test in more than one experiment. With an in-house testing team, corroboration of test results became an important goal.

Tests run: Quickly Deploying Learnings Across Websites

WMG was able to triple the number of web tests ran over a year, taking it from 10 to 38 experiments. The tests also resulted in an average improvement of 5% in conversion rates across 15+ websites.

Moreover, insights from the successful test results allowed WMG’s UX team to quickly deploy the learnings on their other websites. Here are the key scenarios where WMG had the maximum impact on its conversion rates using VWO:

Heatmaps and Visitor Recordings Improved Website Conversions by 2.7X

The Heatmaps and Session Recordings capabilities provided WMG with an understanding about the visitors’ intent and behavior. With the help of the VWO, WMG optimized 11 websites for a variety of artists. This effort furnished conversion rates 2.7X better than industry standards, based on the data available with them.

Leverage VWO Insights to identify the elements of your website that catch visitors’ attention and the areas of friction. Start your free trial today!

Splash Pages Saw an Average Improvement of 4% in Conversions

Splash Pages are an important source of revenue for both WMG and its associated artists, as these are the pages visitors see first, before being given the option to continue to the homepage. While analyzing splash pages for various artists, the UX team noticed that conversion rates were lower than the industry average. As a countermeasure, Brandon decided to experiment by redesigning the Splash Pages.

The redesigned Splash Pages ran 3 A/B tests before going live. Each test result showed an improvement in the conversion rate.

Contextual CTAs Improved Conversions by 18%

Another valuable discovery Brandon and his team were able to make using VWO’s Visitor Behavior Analytics was the importance of contextual information accompanying the CTA. By running A/B test based on this insight, they were able to improve conversions.

By A/B testing the CTAs based on the contextual information, the team was able to develop a standard series of tests. After replacing the generic messages accompanying the CTA with contextual messages, the conversion rate improved from 14.0% to 18.6%.

Brandon and his team also experimented with placement of signup forms on their website. The variation saw an improvement of 12% over the control version. The control had a conversion rate of 6.2% compared to 18.4% of the variation.

Conclusion

WMG is an example of a thriving experimentation-based organization, where every effort toward improving user experience and conversion is derived from a systematic and planned approach.

With the sheer number of tests that resulted in positive results for WMG, Brandon and his team were able to develop a guideline to web testing for higher conversion and improved user experience. This was made possible by combining all the observation and hypothesis stored in VWO.

VWO lets you easily discover insights, build an experimentation roadmap, test ideas and, engage visitors, all within a single platform. Start your free trial today!