August 2005 Optimizer Newsletter Now Online

The 2005 edition of our conversion rate optimization newsletter, dubbed "The Optimizer," is now online here. If you not not receive it via email, check it out. If you would like to subscribe, please visit http://www.vertster.com/newsletter/

Katrina in your test results?

Even if you live nowhere near the gulf coast, hurricane Katrina could be having a major impact on your test results. This is a classic example of a huge environmental noise factor, something that can shift the buying sentiments of an entire population. If you live near the affected area, or cater to a local business, this will be obvious.

But what about us who live hundreds or thousands of miles away?

The press coverage has been more than constant- its been downright transfixing. Anyone who knows someone living near New Orleans or who has been through a hurricane probably spent a little extra time watching the news today... and a little less time surfing the web or "taking care of business." Not to mention the pervasive online news coverage and blogs pulling eyeballs and dollars away from our sites.

To make matters worse, the threat of spiking gas prices has many people hitting the brakes on purchase decisions. We noted a significantly lower conversion rate today on at least a couple of our "key indicator" tests. It seemed more like a Saturday than a Monday.

We use the term noise to refer to any uncontrollable factor that can affect the outcome of our tests, whether it is a positive or negative change. Split testing helps to disperse the effect equally within the test battery, but it doesn't always eliminate an uncontrolled bias from entering the test results.

Take for example, a site that sells flood insurance. If this was you, today was probably a great day for sales, perhaps even record breaking. Would today be a good day to get repeatable test results? I think not. The winning creative today probably would be the one with the shortest and easiest order form. Today, many people wouldn't need much convincing. Normally, the best pulling message might be a completely different appeal altogether. Basing your test results on today's data, even if statistically significant, would lead you to erroneous assumptions.

More from our Blogged Optimization Campaign

We are getting really close to releasing some results from this thing. Its been running for a while, and is setting up nicely. In the meantime, here is another brain challenge for you: We tested three different images.

1. Displays the free DVD that you will receive.
2. Shows a successful and happy couple.
3. Shows a successful, smartly dressed business woman.

Which do you think would result in the highest response rate?

Just found this!

Bob Bly has an interesting blog post this morning, asking folks to choose which of three direct response offers would get the highest response. Check it out and cast your vote!

Back from SES for a week, and I am still buried

Well I returned back to Salt Lake City last week, but I have been completely buried from spending 3 days outside the office. Now my folks are in town for a couple days, which is a further distraction from giving you guys the quality blog information you have come to expect. Thus, I have not posted an update to the blog. That will change, I am forcing myself to get with it here!

We have some exciting results to share on the "Blogged Taguchi Optimization." I hope to add another posting today with some more information on that one. Also, I promised a special offer for the folks that attended the Landing Page Testing session at search engine strategies, and we still plan to make good on that offer. So keep posted.

Update from Search Engine Strategies

I am still out here in sunny San Jose, at the 2005 Search Engine Strategies conference. So far its been a great show, with huge numbers of people here, and lots of great sessions. The landing page testing and optimization session was a huge success, with standing room only.

Last night at the Google Dance I got the opportunity to ask many folks that attended what their thoughts were, and all said it was by far the best session so far at the conference. I was thrilled to hear this, as we put a lot of work into it, both developing the content and practicing/ rehearsing. It sure paid off. Getting the remote voting devices made the session truly interactive, and kept people "in the conversation."

Live Taguchi Optimization- it's back

Well its been a few days since I posted an update on this project. We are now running and starting to collect results. However, before I show those (this will come on another day,) I wanted to focus on a couple of the interesting factors we are exploring. One of the most interesting to me is the form position.

The response form is always important, since it is the way conversions come in. For this test, we are actually looking at using the same form, in three different locations- in the right column, the left column, and underneath the text body. I have always believed that putting the form above the fold was important. On most landing pages I have designed, you can see this. The right column is a personal favorite of mine, using a 65/35 lineup.

Which form position do you think is best? I bet you never thought about testing it :)

A/B Testing Secrets Revealed!

Dr. Flint McGlaughlin and his crew at MarketingExperiments.com did a real nice job at providing an overview of split testing tactics today. You can listen to the audio portion here and view the visual materials that went along with it.

The webinar started off with an audience interactive "vote for the winner" comparison of three different landing page approaches. The vote was a bit tricky because there was little background given on the circumstances surrounding the page. I won't give it away here, but I will say that I guessed incorrectly which was the "winner."

I highly recommend if you are new to landing page testing, or want to pick up some tips on A/B testing, that you give this a listen. It runs just over an hour, and is packed with tips and real world experience. The only thing I would add is that you run your tests in 7 day blocks, especially if your test lasts more than 14 days. This provides an equal helping of weekdays vs. weekend days. Running a short test that does not balance the days adds a significant noise factor and can produce unreliable results, even if your sample size is large.

SES San Jose is next week

Next week at this time, I will either be- testing the handheld voting devices we are using for the Landing Page Testing and Tuning session, hanging out at the Yahoo! party, or sitting by the pool in San Jose.

The operative word in that sentence of course is San Jose! Search Engine Strategies is almost here already, and I am pumped. We're gonna put on a great session on Tuesday on landing page testing, and I can't wait to meet some new faces. If you want to hook up for a handshake, beer, coffee or otherwise, shoot me an email at scott@vertster.com

Also, keep your eyes on the blog, as I will be posting live updates throughout the event, including a link to the powerpoint from the presentation.