A Guide to A/B Testing Tools


BEST PRACTICES SERIES

Article ImageIn this age of analytics, marketers are data detectives. A/B testing can help resolve many of their quandaries related to online marketing.

But, first, what is A/B testing? It is a process in which you choose the best-performing version of a webpage, by randomly displaying different versions of your site to visitors and assessing the performance of each variant against a desired metric (such as clicks or sign-ups). You can test by tweaking one page element (such as headline, call to action, or image) at a time, or you can test changes to several page elements all at once (the latter is referred to as "multivariate testing"). Thus, when you use A/B testing, you are not flying blind. You're letting data drive your design choices and decisions. Think of A/B testing as "the scientific method meeting online marketing."

Not just for webpages, A/B testing use cases span email campaigns, banner ads, mobile apps, and other marketing scenarios in which even a small increase in conversion rate significantly moves the needle on business outcomes. 

Strictly speaking, A/B testing techniques are not new. However, until a few years ago, the complexity of implementation meant that only companies with large teams of marketing analysts, data scientists, and developers were able to effectively use them in practice.

A/B Testing Tools

The good news is that, today, you have tools that range from simple to sophisticated, which let you leverage A/B testing without having to understand the complex statistical science behind it. Examples of A/B testing tools include AB Tasty, Adobe Target, Google's Content Experiments (Google Analytics), Maxymiser, Monetate, Optimizely, Qubit, SiteSpect, Unbounce, and Visual Website Optimizer. The breadth and depth of functionality offered by these tools vary, but here is what you can expect to find:

• Visual editor-This is used to create the variants without having to write HTML/CSS code. An exception is Google's Content Experiments, which is free, but does not have a WYSIWYG editor.

• Multivariate testing-This is useful when you want to change several page elements, but the test requires more traffic for accuracy.

• Multipage testing-This is used to test changes to a multistep shopping process in an ecommerce scenario.

• Segmentation-This is helpful if you want to run tests on certain visitors or understand if variants perform differently for different visitor segments.

• Automation-After the test is complete, the tool can automatically display the best-performing variant going forward.

• Mobile app A/B testing-Some of the previously mentioned tools let you A/B test websites and mobile apps. There are also specialized tools for mobile app A/B testing, such as Amazon (in beta) and Splitforce. Using such tools, you can A/B test mobile apps without having to resubmit an updated app to the app store.

Selecting an A/B Testing Tool

In addition to core functionality, here are the non-functional considerations in selecting an A/B testing tool:

• Ease of use-The number of experiments (i.e., A/B tests) can be quite large. If the tool is easy to use, nontechnical users can run the tests on an ongoing basis without much support from the IT department. Also, does the tool fit into your existing web development and update mechanisms, or do you have to make significant changes your current processes?

• Client vs. server-The tools use either client-side or server-side scripts to serve the appropriate variation and track performance. Each approach has its pros and cons. Client-side tools (e.g., Optimizely and Monetate) are simpler to implement but may involve some browser compatibility or performance issues. Server-side tools (e.g., Adobe Target and SiteSpect) offer flexibility and more integration options.

• Integration-Large websites are typically powered by several enterprise systems, such as web content management (WCM), marketing automation and personalization, and analytics tools. If A/B testing tools can integrate with such underlying systems, you cannot only conduct more fine-grained A/B tests but also can deliver more personalized web experiences to your site visitors.

Conclusion

The advertising guru from the Mad Men era, David Ogilvy, said, "Never stop testing and your advertising will never stop improving." We did not have the tools then, but today, smart marketers don't have to sweat the small stuff-they can just A/B test.

Done right, A/B testing can serve as your real-time focus groups at scale. However, keep in mind that it is more suited for incremental and iterative improvements; it's not meant to provide you with revolutionary insights or radical leaps. A/B testing would not have led you to invent the iPhone-but once invented, it can tell you the colors iPhone customers prefer.   

(Image courtesy of Shutterstock.)


Related Articles

Two weeks after its 2013 launch date, Electronic Arts, Inc. sold 1.1 million copies of its "SimCity 5" game, with digital downloads representing half of all sales. President Barack Obama's 2012 campaign website increased donation conversions by 49% and, ultimately, raised an additional $500 million. And Amazon famously changed its checkout button 6 years ago, increasing site revenue by $300 million. What's the common denominator in all three success stories and in countless other case studies across the internet? Each conducted A/B tests, which produced phenomenal results. And they're far from the fringe minority: 75% of strategic phase marketers rely on A/B testing to learn about customer behavior, per MarketingSherpa's 2012 "Website Optimization Benchmark Report."
According to Gartner, Inc., web content management (WCM) software has outgrown being only a tool for creating web content. In addition to serving content, WCM vendors have set their sights on helping companies manage customer experiences and are delivering software to help them realize that ambition. This shift is due to the fact that consumers are harder to reach and more discerning, according to Mark Floisand, VP of product marketing for Sitecore.