How to Choose What to Test
Testing is an iterative process
Taking your time is crucial. Tests require an adequate run to really determine what is and is not working. After a sufficient amount of traffic has rolled in (figuring out just how much is needed is a skill itself), the data becomes clearer and less susceptible to noise (GA experiments will even stop the test once they’ve reached certainty!).
A/B testing requires patience and repetition. If you don’t obtain significant results the first few times, keep testing. Focus on creating a large number of smaller tests and refining your testing each time with the things you’ve learned. Did your hypothesis hold up or does it need tweaking? What did the variations that performed better have in common?
The best thing about A/B testing? It always produces a result. You can run a test for days backed with a great hypothesis and backed by concrete data, and it could still return next to nothing in terms of actionable data. However, that lack of a result will teach you what did not work, and give you more insight into your customer’s behaviour.
This information can now be used to change your assumptions and come up with a new hypothesis, redefine your problem statement, and look for new solutions in different areas. You can always be learning, refining, and optimizing. Don’t fret about it!