By showing different variations of a product or feature to segments of your audience, you can quickly learn what works and what doesn’t. There’s a reason that every major tech company such as Google, Apple, Uber, etc. Use split testing as a part of their design and research process. Done correctly, you can learn about your customers rapidly and increase revenue. When done wrong, split testing can become an expensive waste of time.
Here’s what happens a lot when someone reads an article on ConversionXL and decide to start split testing:
When you test this way, you’re missing out on a majority of the value split testing provides. You need to approach split testing with the right mindset to get good results.
The more you understand your customers, the better you can serve them. To learn the most you can, you need to approach testing with a scientific mindset. Instead of saying “Let’s try something,” or “Let’s pit Tom’s idea against Sarah’s and see who wins,” Start with a hypothesis. Say “If we make this change, I think this effect will happen because of this reason.” When you approach testing this way, every test will be a win regardless of income:
Before you run a test, ask yourself these questions:
Since we are testing, any reasonable hypothesis warrants consideration. If there is a justification, go for it. If you have a reason puppy GIFs would increase conversions with your audience, go for it. Ideas should come from anywhere in the company, not just the design team, marketing team, management. Employees working closer to customers and the product will often have insight other departments don’t. When coming up with test ideas, use data and research to back up your points, but keep an open mind.
Don’t take test results personally. Don’t use the phrase “my test” or “my idea.” Talk about your test in the third person. You’re a scientist, remember? Treat your thoughts with a healthy, cold, logical distance. You’ll need to because…
Most of the time, split tests lose more than they win. You’ll have ideas that you think are surefire winners that will bomb horribly. Think of it like baseball. The best hitter in the 2016 season missed 65.2% of his at-bats. Losses are temporary. Wins are evergreen.
Often the biggest challenge isn’t what to test or how to test, but getting buy-in that testing is the way to go. Split testing can be uncomfortable for some organizations; you want to help alleviate that awkwardness by highlighting the benefits. Even though you aren’t taking things personally, it’s still nice to let yourself have a moment to enjoy when a test goes well.
The more people that benefit from your testing, the better, When you get results, win or lose, share them with the rest of your team. Marketing, support, design, and management can all benefit from the data you can gather via testing. Keep sharing an open and company-wide process.