Two weeks after its 2013 launch date, Electronic Arts, Inc. sold 1.1 million copies of its "SimCity 5" game, with digital downloads representing half of all sales. President Barack Obama's 2012 campaign website increased donation conversions by 49% and, ultimately, raised an additional $500 million. And Amazon famously changed its checkout button 6 years ago, increasing site revenue by $300 million. What's the common denominator in all three success stories and in countless other case studies across the internet? Each conducted A/B tests, which produced phenomenal results. And they're far from the fringe minority: 75% of strategic phase marketers rely on A/B testing to learn about customer behavior, per MarketingSherpa's 2012 "Website Optimization Benchmark Report."
A/B TESTING EXPLAINED
A/B testing is the term used for randomly experimenting with a control variable (A) and an experiment variable (B) for the purpose of statistically testing a hypothesis. As it applies to online site design, it's the process of testing and comparing two similar versions (A and B) of one or more web/mobile pages to determine which ones perform better and produce better conversion rates-which can include sales, hits, leads, and click-throughs-among randomly sampled, but similar, visitors. A/B tests can compare like elements, content, or designs, but not necessarily; sometimes, a larger discrepancy between the items being tested can help with ruling out a specific approach or a departure from the norm.
Today's A/B testing technologies afford a nearly limitless ability to experiment with various approaches for providing useful content and navigation to users. Tests are no longer limited to a single page. Websites can now test functionality that exists on many pages throughout a site, such as the presentation of assisted navigation that allows visitors to filter their results while searching for content or products.
Many sites now perform multivariate testing, which is a more sophisticated type or variation of A/B testing that lets users test individual elements within a piece of content or design in order to understand the relative impact of each element, or combinations of elements, upon conversion. Multivariate testing can be most useful as a discovery mechanism. You can test a few variations of each of the elements (such as imagery or messaging) in order to see what element has more impact on conversion. It's desired by many because it produces more particular results and more intricate data. "In the past, it could take several weeks, possibly even a month, to coordinate all the components of an A vs. B test. Today, it can be set up and perfected in a day or so," says Jason Parks, owner of The Media Captain.
Bob Dufour, president of Fusion, says A/B testing allows today's digital marketers to change things in an evolutionary way and, ultimately, be more responsive to their customers. "A/B testing has evolved from relatively simple to an increasing use of multivariate designs that will allow you to have multiple changes or variations per test," says Dufour. "Instead of doing 10 A/B tests, you can do one or two multivariate tests, learn more quickly, and predict outcomes better."
(Image courtesy of Shutterstock.)