Skip to the content

What is A/B Testing?

The A/B Testing Primer For Digital Agencies

If you want to succeed in any area of online business, you’ve got to be constantly adapting, tweaking, and fine tuning the features of your website to optimize the visitor’s experience. Even sites bringing in droves of traffic can still be improved upon. As a digital marketing agency, you most likely understand the value of making visual or usability improvements, as well as the value of plugging holes in your sales funnel; after all, the end goal of a website is not about just bringing in more traffic, but ensuring that each visitor has a high probability of making the right decision once they get there.

Conversion Rate Optimization (CRO) is the science of using analytics and user feedback to improve the performance of your website. Its end goal is increasing the % of visitors that complete a defined goal (for example, buying a product or giving an email address). Thanks to decreasing costs in both money and effort (with a CMS like Umbraco, you might spend nothing!), CRO is all the rage these days.  

The issue is that it can be relatively easy to see that changes need to be made, but determining which changes they should be and convincing decision makers to give the greenlight can be a bit tricky. Imagine how great it would be if you could just ask each visiting customer what they’d like to see and then make those changes with the peace of mind of knowing that it’s what works best for them.

Behold, A/B testing. A/B testing (also known as split testing) lets you create variations of a page and test them on a sample of visitors to see which one performs better in regards to a predetermined goal. One part science and one part social experiment, running A/B tests puts the decision of which improvements to make to your site in the hands of the people who matter the most: your customers. Changes to your site are expensive, time-consuming, and a bit cumbersome. In short, they’re dangerous. Making changes on a hunch is almost always going to result in wasted time and money.

The beauty of running A/B tests is that they let you narrow down your customer experience to exactly what visitors want to see without big costs or commitment .

The goal of A/B Tests: give the people what they want (or what positively influences behaviour the best)

Your first goal in terms of your website is to attract visitors. Once you actually have a steady flow of traffic you can begin to work towards your next big win: getting more interaction from them. Indicators of better customer interaction might be a:

  • Higher conversion rate- your conversion rate is simply the % of customers who complete the desired action of your web page. They are the visitors to your site that take their relationship with you to the next level. That could be signing up for a mailing list, purchasing a product, or downloading an ebook.
  • Lower bounce rate- once someone lands on your page, they have the option to either stay or leave (bounce). If they click back, navigate away, or close out, then they’ve left without doing anything. The % of visitors that land on your page and immediately leave is your bounce rate. The lower the better!
  • Session duration-session duration is valuable for gauging how pleasing (and relevant) a site is to a visitor. Sites with a better user experience keep customers involved longer. This is just the average total time spent on all pages during a single visit.

How do you increase interaction? By providing unique, valuable, and engaging content and making your message very clear. The way to create a better overall experience is by giving your customers what they want to see, and what better way to do that than by finding out exactly what matters to them the most. A/B testing provides a way of directly consulting them to see what makes them take the desired action.


Next up: How does A/B testing work?