A/B Testing Quick-Start Guide, Part 2: Setting Up Your First A/B Test

By Rob Canning

We're building upon the basics of A/B testing by explaining how to quickly and easily get started with testing on your site.

As a small business, you have a ton of competing priorities, and finding time for A/B testing and optimization of your website can be challenging.  Part 1 was an overview of A/B testing, its value, and how to select pages for testing.  Part 2 builds upon the basics by explaining how to quickly and easily get started with A/B testing on your site.

RelatedA/B Testing Quick-Start Guide, Part 1: What is A/B Testing?

A/B Testing Software

The number of Testing and Optimization software options has increased significantly over the past several years.  In the mid-2000's, the well known and widely used testing software cost around $10,000 per month and required at least half of a full time employee's time to administer and operate.  As testing has evolved and become more commoditized, these days it's not uncommon to pay as little as $49 per month for a full featured and easy to use testing platform.

Two great platforms to try out are Wingify's Visual Website Optimizer (VWO) and Optimizely.  Both are very easy to use, and both offer a full set of features for a low monthly price.  After initial setup, these platforms require little or no IT involvement to set up new tests.  Even initial setup is relatively easy for anyone with basic HTML skills: a single code snippet needs to be pasted into your website's header so that it will render on every page of your site.  Both VWO and Optimizely have free 30 day trials. If you'd like to explore a comprehensive list of testing software available today, this site has done a great job of compiling an unbiased list: http://www.conversion-rate-experts.com/split-testing-software/

A note on Google Analytics' Content Experiments: Google used to have a free A/B testing tool called Google Website Optimizer that has since been retired.  Google now offers testing and optimization functionality through its (also free) Analytics software.  The new version of their software requires that you create multiple variations of pages, and then it splits traffic between the pages and tracks the results.

Setting Up Your First Test

You're ready to begin testing when:

  • The testing software code is in place (see above)
  • You've selected a page or pages for your test (see Part 1)
  • You've chosen elements that you want to test (see Part 1)

Steps

  1. The first step is to use the testing software interface to create a test version of the page that you want to test.  VWO and Optimizely both have drag and drop interfaces that allow you to easily modify your page.
  2. Next, set the traffic split to 50% control (your original layout) and 50% test
  3. Turn on the test through the testing interface, and monitor it regularly to get an understanding of early performance and whether statistical significance has been reached (see Part 1 for more information on statistical significance)

Related: The Top 4 Tips for Dealing with Marketing Complexity

Iterating

It is important to keep testing the same page until you've exhausted ideas and your gains begin to plateau.  Inevitably, some tests that you're 100% convinced will win (i.e. provide a boost to your metrics) will actually perform worse than your existing page.  This happens often in A/B testing.  They key when this happens is not to abandon the test but to stay the course and iterate on the failed tactic or try something else.

Staying the Course

Lastly, the biggest mistakes with any new or existing page on your site are a) not optimizing with A/B testing, and b) getting discouraged by early underperformance and shifting focus to other projects.  Don't get discouraged if your early A/B testing efforts fail!  It's very typical if the first couple tests either lead to a decrease or have no effect.  With a bit of time and practice, you'll be on your way to increasing your key metrics in no time.

(Image: nokhoog_buchachon via freedigitalphotos.net)

Login to Business.com

Login with Your Account
Forgot Password?
New to Business.com? Join for Free

Join Business.com

Create an Account
Sign In

Use of this website constitutes acceptance of the Terms of Use, Community Guidelines, and Privacy Policy.

Reset Your Password

Enter your email address and we'll send you an email with a link to reset your password.

Cancel