receives compensation from some of the companies listed on this page. Advertising Disclosure


The Comprehensive Guide to A/B testing for E-commerce

Kristina Azarenko

You’ve probably heard a lot about A/B testing, haven’t you?

You might even know that this is a technique of testing changes on a page to find out the most effective ones.

But there are questions that still need answers.

Follow the article and you will fill the gaps in your knowledge about A/B testing.

The main questions about A/B testing answered in this article are: why? what? and how?

Related Article: An Inside Look at the Importance of A/B Testing in Paid Search

Why A/B Test?

A/B tests, or split tests, are created to:

  • Increase conversions
  • Improve user experience (to increase conversions)

As you can see, the purpose is actually predictive, while the results of such experiments can be surprising. Sometimes even minor changes in a page copy can make a huge difference.

Types of A/B Tests

There are three main types of split tests depending on the amount of changes you are testing:

1. One Element at a Time

2. A Few Elements at a Time

3. New Layout

Important Note 1: According to best practices, the elements should be tested individually. For example, if you change product description and product images, you don’t know which of these changes is responsible for the results of your test. Therefore, it would be better to test different descriptions and images separately.

However, sometimes one needs to see how a completely different layout will influence the conversions, time on page, number of adds to cart and other metrics. In this case, it’s ok to change many page elements and test them as a unity.

Important Note 2: Though traditionally A/B tests are conducted with two pages, you can use as many test pages as you want. For example, you can test 3 different product descriptions: one original and two variants.

What to A/B Test?

Here are a few ideas that will help you start with split testing in your store.

1.  You can test your call to actions to find the one which has more influence on your customers:

  • Add the word “free” (if it’s really free).

  • Add the value of the product.

  • Add the period or quantity limits.

2. Test product description. You can test its style, formatting, etc.

3. Test layout of the elements on the page.

4. Test images to find the most appealing ones.

5. Test button colors.

The trick here is that there is no single color that fits all eCommerce sites. One site can increase conversions using green “Add to Cart” button, another store can change the button color from green to red and get more sales. The rule of thumb here is to use a color with high contrast to the background as seen on the right image.

6. Test your forms.

Here is a quick suggestion for you: review your forms and leave only fields that are really needed. The fewer fields a form has, the more people are likely to fill it in.

Related Article: 10 Best Practices for Paid Search (Part 1)

How to A/B test?

There are some good testing solutions available, both free and paid.

1. Google Analytics experiments.

This is the most widespread tool since almost each web shop uses Google Analytics. Creating experiments is quite simple provided you can easily duplicate a page and have an opportunity to insert the experiment code to the <head> of the original page. The problem is that not all platforms can provide you with these options without customization.

Additionally, even if you can duplicate a page, you must be sure that the original and test pages are identical (except for the element you are going to test, of course). For example, user reviews should also be copied as there is no point in testing anything if your test page is lacking the customer reviews the original page has.

I’ve created many Google Analytics experiments for Magento sites. Frankly speaking, it may become a real pain without proper developer help. If you go for that solution, you can use this guide.

2. Optimizely

This is a great platform for testing your pages. The main advantage of it is flexibility: you can create page variations in your Optimizely account, configure your test setting, and those variations will be automatically shown to the visitors. Optimizely is based on a monthly subscription. They have interesting case studies that may also give you some ideas of A/B tests for your store.

3. A/B testing extension for Magento

Just like Optimizely, this extension is a separate A/B testing system which is not connected with Google Analytics experiments. It uses native Magento product attributes to create tests.

4. A/B testing integration for Shopify

This module also lets run split tests on the basis of Shopify product attributes which sounds quite easy.

Magento and Shopify are ones of the most popular eCommerce platforms so the chances are high you are using them. If not, you can choose universal solutions listed above or find specific modules for your platform.

A/B Test Conditions

Condition 1:

Before starting any experiments, you should determine which action will be treated as a conversion. You can test micro (e.g. newsletter sign up) and macro conversions (e.g. order completion). This action can be called a goal and it will measure the effectiveness of each page in the experiment.

Condition 2:

You will need quite a long period of time, at least 2 weeks. As I’ve said above, the results may be unpredictable, so you should make sure you’ve seen a whole picture before stopping your experiment.

Condition 3:

Your test pages should receive sufficient amount of traffic. There’s no point in testing anything with 30 visitors in a month, you won’t get reliable results.

Condition 4:

Don’t create promotions on the test pages unless this is the purpose of your experiment. You won’t get pure results if you hold a sale or something of that kind.

Related Article: 3 Words That Will Instantly Boost Call-to-Action Conversion Rates

Final thoughts

A/B testing is very interesting; you will hardly find a final point in it. There is always something more to test. Create a split test schedule and plan your test beforehand. It will let you think over variations you’ll need to test and goals you want to achieve.

Image Credit: NanoStockk / Getty Images
Kristina Azarenko Member
Kristina pays careful attention to the latest marketing trends while developing detailed strategies for site performance improvement. She is a marketing manager and SEO specialist at Amasty, a company that develops professional Magento extensions which make the lives of Magento store owners easier.