Split Testing Strategies
Split Testing on your home page... Good or Bad Idea?
The home page. It's the front door to your website and the page that might have the most influence on your total sales. It is almost always the first place new customers want to test. Should you start here though? This is one of the most frequent questions I get, and the answer depends on a number of factors.
Like a line in the sand, online marketers seem to polarize and take one side or the other. I certainly have my own beliefs on whether the home page is a suitable place to test. But I have written this article to help you make a smart decision. The fact is, the answer is usually dependent on your site, your offer, and other factors.
One thing is almost a given: The homepage is without a doubt the most difficult page to test on 99% of sites.
Reasons homepage testing is a Good Idea:
There is a lot of traffic. Your home page probably gets more unique traffic than anywhere else on your site, and certainly gets more site entries. That makes it ideal for testing, allowing you to reach a point of statistical validity faster than anywhere else... or so it would seem.
It can be influential. Your home page could have the most influence on potential buyers. The home page normally contains a good deal of promotion, and this makes it ideal for testing. With many people landing on your home page, it makes a lot of sense to try and improve this "first impression."
The home page gets a mix of traffic types. The fact that people are coming into your site from all walks of life (figuratively speaking,) means that it gets a very broad mix of traffic, ideal for perfecting an offer or creative. Incidently, this is also one of the negatives associated with your home page. Read on...
Reasons homepage testing is a Bad Idea:
The traffic is highly variable over time. What works well one month, may not work so well the next. The major search engines shift their index on a monthly basis (or more frequently.) The mix of traffic you get today, may be completely different from the mix you will get tommorow, as the keywords that customers use to find you change.
The current keyword audience may respond very well to Offer A, and your testing will indicate that A is a superior performer. However, when Google updates their index, suddenly all the traffic that was buying on your site, heads to a competitor who has taken over the top spot. Your testing could not have planned for this, and even though your results were statistically valid, your NEW audience might actually respond to Offer B better. Of course this could change in another month, but it is the nature of the home page... difficult to make judgements on.
There are more than just prospects hitting your homepage. This is one of the biggest problems facing homepage testers. Lets think about a few of the possible objectives people landing on your homepage might have: competitors researching your company, job hunters, press looking for a story, current customers looking for your phone number or other support information, and this is just a start.
This is what we call a random noise factor. It is unpredictable and random in nature. Some days there could be a lot of it, some days not so much. The net effect is that is waters down your test results and makes them harder to see.
Even if you have achieved "statistical validity" in a classic sense, it may be hard to trust your results. More likely, unless you have a huge margin between your winner and control, getting statistical validity may be hard.
Determining the goal can be a challenge. Most homepages present a mishmash of user objectives. There is the newsletter signup, the whitepaper download, the sales contact, and the buy button. This represents a problem. What if you optimize the whitepaper download at the cost of sales? Did you really end up with a better page? How do you measure a conversion?
Any change you make will affect your search engine rankings. Using smart testing software, you can avoid impacting your search rankings during the test phase. But the time will come when you decide to make changes to your text, images, or layout permanent. When you do this, you will see an impact on your site’s search engine visibility. Since the homepage usually is the most important from a search perspective, making changes can be risky.
As you can see, there are numerous considerations in testing and optimizing your home page. We suggest that if you are just starting out with testing, focus on a more targeted page inside your site first. This will help you learn the ropes. Once you've started to understand some of the challenges online testing presents, then consider moving to the homepage.
