Skip to the content

A/B Testing Tips Summarized

Keeping these best practices in mind will help you run an A/B test that gives back more than you put in. If you want to get big results, then just follow these best practices, keep trying, and always be learning. Just to summarize:


  • Create a hypothesis
  • Create a problem statement
  • Define a goal
  • Define your metrics of success
  • Analyze existing data to find patterns that help you refine the problem statement
  • Focus on the aspects of the page with the most potential to produce results (calls to action, buttons, headers, placement, etc.)
  • Keep an open mind and explore broader solutions before focusing completely on one
  • Run tests for an adequate length of time and subject them to the same variables
  • Interpret the results in the context of your hypothesis, change your assumptions, understand your customers better, and apply what you’ve learned to refine your future tests
  • Be humble (your website isn’t perfect already)
  • Don’t fret if your test doesn’t return results at first


Now that you’re an expert at running and evaluating your tests, you’ve got to do the legwork on the technical side to actually set it up. This is often a major barrier to successful testing, and one that discourages a lot of companies from running tests altogether.

The key to successful A/B testing isn’t making changes and waiting for traffic to roll in, it’s actually being able to plan and evaluate the results through the right lens. Approaching your test like a science experiment (complete with problem statement and hypothesis) as well as keeping an open mind and exploring the possibilities are like building a strong foundation for a home. Use what you learn from each test to further refine your hypothesis and your site in general (remember, testing is an iterative process!).

The great thing about A/B testing is that there are tons of tools that help take your tests to the next level. Now that you know how to approach a test and run them the right way, you need to know a bit more about some of the awesome tools at your disposal. It’s also important to find out what exactly makes them difficult to integrate into Umbraco and what the solution to those woes are! But first, you should learn more about the rocky relationship between Umbraco and A/B tests, and why it makes life so difficult sometimes. Check it out here

Continue to Part III: A/B Testing in Umbraco