7 Do’s and Don’ts of Website Optimization Testing
WARNING: Website optimization testing is extremely addictive. Sitting behind the scenes, pulling the strings, and seeing the effect of positive change is overwhelmingly rewarding.
So now you’ve committed to testing your website. Where do you start? What do you test? While you’ll ultimately have to answer these questions for yourself, below are 7 Do’s and Don’ts to help guide a successful testing strategy.
- Do Test Properly: So you tweaked your homepage and sales shot up 50% from the prior week. Success? Not really. There’s too many moving parts here to consider. Did you run a sale during the same period, or send an email? In order to determine whether a new version of a page outperforms an old one, you must run both pages at the same time under the same conditions, and split test them using software such as Google free website optimizer. Before you advance any further with your testing strategy, get acquainted with Google’s powerful testing tool.
- Don’t Be Afraid to Fail: In testing & optimization, failure is success. Although you may be disappointed that the new landing page you created underperformed against the old one, had you not tested it, you would have caused unknown damage to your sales.
- Do Be Patient: Let your test run until you have a statistically significant winner. It’s tempting to end a test early when your favored version is winning, but fight the urge. Once your test has started, set it and forget it.
- Do Use Testing to Settle Disagreements: Testing is the ultimate equalizer of opinion’s, especially those of the HiPPO’s. Rather than argue endlessly with clients or managers about what route is best, offer an unbiased test of each option. I can’t overemphasize the value of this approach. Eventually, you will reprogram the thinking of your company from “let’s do it!” to “let’s test it!”
- Do Prioritize What You Test: When determining what to test, prioritization is crucial. Focus on the highest traffic pages first, those which are higher up in the conversion funnel. Bryan Eisenberg of GrokDotCom suggests testing the following pages first:
- Your Top 5 High Bounce Rate Pages
- Your Top 5 High Exit Rate Pages
- Your Top 5 Lowest Time Spent Pages
- Your Top 5 key pages (i.e., checkout, cart, registration, top product)
- Don’t Test Too Many Variations: While you can run a multi-variant test that allows you to test dozens of different versions of a page at once, such a test will require too much time to determine a statistically significant winner. Only test variations that you have reason to believe will impact the bottom line.
- Do Project out Benefits: After you’ve found a clear winner in a test, estimate the value of that change over a future time period in order to justify testing to management. Like other marketing activities, you’ll need to prove the ROI of your testing activities. Suppose a change to your site navigation increase your conversion rate by 10%. Calculate what that means for your business over a period of a year, and communicate your success to stakeholders.
Optimization testing separates good websites from great websites. Following the tips above will help you attain your testing goals. Have your own testing advice? Leave a comment below.
| Share |
Related Posts:


April 22nd, 2009 at 7:48 am
Letting the test ride out is a good point. I am running a test now and the results have been back and forth a few times with each version of the A/B test, looking like it was greatly better than the other. But now the tide has turned toward the newer page and I will wait and see what happens.
April 23rd, 2009 at 2:51 pm
Sound advice. Thanks for a quality post with good info.
May 4th, 2009 at 5:41 am
Thanks for these good advices…..patience can often be a problem – and I completely agree that faliure is af succes.
May 7th, 2009 at 9:27 am
I just created an ecommerce blog and I will be needing this advice to help me stay on the right track. I do have a knowledge in optimization but it is still best to get some help from others.
May 21st, 2009 at 7:35 am
Justin,
I love #4 – Do Use Testing to Settle Disagreements.
You can’t argue with valid results and it certainly does lead to more interest from others in continuous testing when they see either their ideas validated or disproved and statistically backed. There’s no shame in not knowing or hypothesizing what is ultimately the best element, page combination,etc. – there is shame in guessing and rolling-out something without testing that should have or could have been tested.
September 29th, 2009 at 10:15 am
[...] is dangerous. By evolving your site, little by little, based on true customer feedback and testing, you’ll slowly but surely build a truly remarkable website. No, it’s not as exciting as [...]