The ABCs of A/B Testing

Page 3 of 3



One of the biggest challenges is testing across multiple platforms. After all, what works on a desktop browser doesn't necessarily work on a mobile device. Consider that the same user may prefer a green button on a laptop but a blue button on a smartphone screen.

"Digital publishers should learn what they can from A/B web testing and apply it to A/B mobile testing," notes Slade. "Mobile testing is more complicated, so you probably won't want to test as many factors on mobile as you do with web. But it's worthwhile running separate tests on both because web and mobile experiences are so different."

Remember that mobile apps "represent a more captive audience, which significantly changes behavior and how testing is accomplished," says Greg Hinkle, CTO and co-founder of Evergage. "[Mobile] users may require time to get used to a new pattern or feature, so tests may need to run longer."

Web-based apps can typically undergo the same type of testing you'd conduct for a traditional webpage. But if you're testing a native app, the procedure is much different and implementing changes is more complicated. "The store providing the app will need to approve app changes, and your customers will need to download the app update, whereas web changes occur instantly. Also, you'll need to find an A/B test provider for apps as opposed to websites," Slade says.


A/B testing certainly isn't going away anytime soon. Experts predict that this popular process of experimentation will continue to progress in the coming years, particularly in key areas such as ease of use. "Early testing tools required advanced HTML and JavaScript coding skills to create alternative experiences, but current testing tools offer ‘visual editors' that allow nontechnical users to make adjustments to a webpage without knowledge of coding," says Nelson. "In the future, tools might use industry best practices and machine learning to suggest page layout options. This could help generate testing ideas and speed up the time it takes to design alternate experiences for tests."

Targeting and personalization should also improve. "The leading optimization tools, today, can automatically adjust which segments of customers receive each test variant," says Nelson. "While this technology is available today, we can expect its adoption and performance to continue to expend in the coming years."

Abisambra predicts a forthcoming era of A/B/C or even A/B/C/D testing. "I can see artificial intelligence supplementing how and when experiments are executed to not waste a single minute and to improve the percentage of experiments that yield a lift or improvement," he says.

Page 3 of 3

Related Articles

Perhaps the only thing that changes more rapidly than technology in today's amped-up digital environment is the terminology used to describe that technology and its impact on consumers--and marketers. One recent example is the advent of the term "omnichannel" marketing, which many struggle to differentiate from another relatively recent term--"multichannel" marketing. Still, those who are most enmeshed in the field say there is a key distinction between the two, and it's one that will have an impact on marketers as they continue to seek ways of having a meaningful impact on the consumers they hope to engage. And, importantly, it's less about technology than it may seem.
In this age of analytics, marketers are data detectives. A/B testing can help resolve many of their quandaries related to online marketing. But, first, what is A/B testing? It is a process in which you choose the best-performing version of a webpage, by randomly displaying different versions of your site to visitors and assessing the performance of each variant against a desired metric (such as clicks or sign-ups). You can test by tweaking one page element (such as headline, call to action, or image) at a time, or you can test changes to several page elements all at once (the latter is referred to as "multivariate testing"). Thus, when you use A/B testing, you are not flying blind. You're letting data drive your design choices and decisions. Think of A/B testing as "the scientific method meeting online marketing."
Sure, posting trip photos, tweeting links, and networking with old co-workers is fun. But online citizens want to tap into the pulse of the world around them, belong to a community, and be recognized. "Social media has evolved into a personal online identity. Before, it was used to connect with friends and update your status," says Jesse Till, social media coordinator for Chatter Buzz Media. "Now, it defines a persona. It's a chance for a person to express and broadcast themselves to the world."