Skip to the content

A/B testing in Umbraco: What You Need to Know

Umbraco gives you a solid base to start from, and with its built-in content managing abilities, it naturally lends itself to creating variations for split testing. There are a few things you should know about A/B testing in Umbraco (at least in its classic and simplest form). 

  • Umbraco is great for creating variations- Umbraco’s main selling point is the ease with which you can make content changes. A few clicks is all you need to change images, content, placement, etc. Creating and managing variations is straightforward. That’s why Umbraco and A/B testing were made for eachother (almost!).
  • No concept of a variation- Umbraco has no concept of content variations, at least not until version 8 has been released. That means any variations (for A/B testing or language translation, for example) are in fact separate pages with their own URL. This might not be so bad, except that sometimes it creates some headaches when trying to run a split test. These pains are usually:
  • Variations in your menus- your menus will start presenting variations to your visitors. Not only does that clutter things up for the user, it actually allows them to navigate to different variations (very bad). Remember, A/B testing is about showing one variation to a visitor at a time! If you want to get rid of them, you’ll need a dev to add some code.
  • Duplicate sitemap listings- your sitemap is a directory that helps search engines better understand your site and all your variations will appear here. Duplicate listings could result in an error or warning in your webmaster console so you will need some functionality to hide them.
  • Cumbersome family trees- duplicating pages also means duplicating the descendents of said pages. This could create a massive maintenance job should you need to make any changes. Even if you make a small change to the variation, the descendents are likely to be exact matches. In terms of SEO and your relationship with the almighty Google, it might not take too kindly to pages that are exact copies. Which brings us to:
  • Duplicate content- the descendents will be treated as separate pages but will also be exactly the same. Google’s algorithm seeks out copied content and penalizes websites that have it. Learn more about that here.


Aside from these issues, A/B testing in Umbraco also means:

  • The redirect issue- try as you might, separate URLs still means dealing with nasty redirects that muck up the works a downgrade your UX.
  • Visual tools detract from Umbraco- once you’ve become familiar with Umbraco, deviating from using tools that you’re comfortable with isn’t very fun. You probably enjoy how easy it is to use Content Picker or Grid Editor. If you use a tool (for example, Optimizely, which you will learn about later), then these visual tools will probably mean the end of your using Umbraco’s tools.
  • Tools require setup- regardless of how you choose to proceed, outside help from a developer is still required to plug in your tool of choice.

Umbraco is a tool with editing capabilities that lend themselves to creating and managing variations for A/B testing easily. In just a few clicks, you can implement on-page changes to test on your visitors. But, as you can see, the nature of the CMS and the A/B testing tools at your disposal often complicate matters, and eliminate some of the advantages that made Umbraco appealing in the first place. Thankfully, there is a plugin that streamlines the A/B testing process, and eliminates many of the drawbacks to testing that might be standing in your way. Next, we will take a look at a few tools that dramatically increase the ROI for your A/B tests but make life a bit more complicated than it needs to be.

Continue to Part IV: A/B Testing with Google Analytics Experiments and How to Make it Better