Time your tests to get best results

Every time you run a test, think about things you can do to reduce noise- or uncontrollable effects on your data. One thing that is often overlooked is fluctuations caused by the day of the week. Think about it. Many sites enjoy significantly more traffic and more sales on weekdays, while others perform best on the weekends.

Web site visitors tend to be very different on the weekends and evening, than mid day during the work week. I have seen a lot of activity and responses come from senior management of very large companies. Rarely during the day. Weekends are when these time-stressed executives get to surf.

Say you run a test starting on Monday. By Thursday, you could have a substantial amount of data, pointing to a clear "winner." The inexperienced tester would begin celebrating prematurely. Along comes the weekend, and what you thought was the winner could end up falling behind.

How to avoid this? Always run your tests by the calendar week. This essentially removes the weekday/weekend fluctuations from affecting your end results.