What is AB Testing?
AB testing in digital marketing is a method for measuring the performance of two or more versions of a
webpage or app. The results of AB testing are then analysed to establish which version works best for a
defined conversion goal.
Webtrends Optimize uses market-leading technology to test any part of a website (such as landing pages or
conversion funnels), and then measures user behaviour of a defined target audience. Using advanced
statistical modelling, Webtrends Optimize can then accurately predict which test result will perform best.
Using AB testing to observe how an existing user experience performs against a new variation eliminates
guesswork, helps you take more calculated risks and make informed decisions.
How does AB Testing work?
As a mini AB testing tutorial the following checklist is a good place to start:
- Start with your data and analyse/identify what area you would like to improve or change in your website
- Identify an element on the page you think will help achieve this and formulate a hypothesis to
prove/disprove your change
- Create a variation based on this hypothesis
- Define your goals, making sure you are tracking the outcome of your test
- Decide who you want to see your AB test eg. all your visitors or just a specific subset (ie. those from
a certain location, or using specific browser, or only return visitors etc.)
- Run your AB test with your new variation against the existing version (control)
Why do we do AB Testing?
The benefits of AB testing are huge and instilling a
testing culture in your organisation can be a key foundation in driving business growth. AB testing
tools can also help create a compelling business case for future development, investment and marketing
campaigns. Below are a few other key benefits of AB testing:
Increase your ROI
This is one of the fundamental benefits for many when AB testing. You should see improved conversion rates,
increased average order value (AOV) and generate higher online revenue.
Lower customer acquisition costs
Your existing marketing campaigns will start to convert better the more you optimise and refine your website
with AB testing, particularly your landing pages.
Improve brand perception/loyalty
Good user experience (UX) builds trust with your customers. By AB testing different elements of your website
you can refine and optimise that experience for your visitors.
Increase your customers lifetime value
Get the bigger picture of your customers' lifetime value by combining AB testing and personalisation
techniques and tracking your customers interaction with your site.
Easy implementation/circumnavigate development pipeline
AB testing tools are simple to implement on your website which will then allow marketers (or anyone you
authorise) to make changes quickly, without requiring a developers input.
What are the different types of AB tests?
The term AB testing is sometimes (incorrectly) used interchangeably with the term split testing. However,
split testing is generally 2 distinct designs of an experience or page. Whereas in AB testing it is usually
only one element of that page you are comparing with an alternative. Some common examples of areas of your
website or app that you can look to AB Test are highlighted below.
Headlines
Your headline is very important and can determine whether the visitor reads your content or bounces straight
off the page/website altogether.
Call To Action (CTA)
The words/tone used, colours, positioning, size etc. of your CTA can make a huge difference to whether your
visitor performs the desired engagement.
Images, Audio, and Video
Do different images/videos engage more than others? Does a video that plays automatically grab your visitors'
attention and keep them on your page more than one where they have to interact with it first in order for it
to play?
Layout/Design
Your content might be excellent but if it's hard to digest your visitors may not stick around to read it. If
you're selling something your visitor wants to purchase but it is unclear how they proceed to do so, you may
lose them without making the sale.
Contact/Booking Forms
Is all the information you're asking for necessary? Is your form overly complicated? Is there a way to
simplify or shorten the process such as AB testing facebook login or postcode lookup?
Content Length
Testing the length of your product descriptions, articles, blogs, FAQs etc. can all have a difference to
obtaining the level of engagement you're trying to get from your visitors.
How else can you use AB Testing?
Your booking/sales funnel and landing pages are always a good place to use AB testing as these pages usually
can have a very direct and immediate impact on your conversion and site goals, given their nature and
objective.
AB testing your website personalisation features combines two very
powerful conversion rate optimisation techniques and will help you identify the right approach specifically
for your own visitors and potential customers.
Similarly, social proof and product recommendations engines can be very useful
features, but AB testing the various approaches to this is important. What type of social proof works for
your website/visitors? Indeed, does it work at all for you?
For product recommendations, how many products should you recommend?
Where should you display these alternative items? What relationship should the additional product have to
the one the user is currently exploring? All and more can, and should, reviewed using AB testing to ensure
the experience you deliver is optimal.
Essentially AB testing is the glue that holds all these Conversion Rate Optimisation (CRO) techniques
together to maximise their efficiency.
AB Testing vs Multivariate Testing
Understanding the different testing methods can be confusing, so what is AB testing as opposed to Multivariate testing (MVT)? Whereas AB testing is simply the choice
between one version of a web page and a different version of a web page, with traffic split between the two,
MVT is different because it allows multiple elements of a single webpage to be changed to establish which
combination delivers the best result.
Both are very valuable features and which one to use depends on what your objectives are and what you are
trying to understand in any given situation.
What are the mistakes to avoid while AB Testing?
1. Invalid or incorrect hypothesis
When AB testing, you formulate your hypotheses before conducting a test. Every decision you make in your test
depends on it, so getting it right is key! Understanding data before formulating hypotheses will allow you
to build good tests and making sure your goals accurately represent success and failure will ensure you run
a fair and valid test with meaningful results.
2. Not testing for the correct amount of time
4 conversions in a variation vs. 1 in the control is a 300% improvement, but meaningless at low volume. You
can even reach statistical significance at very low volume if the observed difference in conversion is big
enough (e.g. 20 vs 0), but experience tells us these are just indicative numbers.
Understanding sample size is key to running tests for long enough, but also not letting them run forever in
hope of a result. Good planning around days also ensures you catch a variety of users who may behave
differently – e.g., late night browsers vs. Sunday morning users.
3. Not re-running tests
Too often, marketers will only test once and believe the first result they receive. However, there is a
possibility of a false positive for statistical significance – we’re all navigating probabilities.
Therefore, it is important to retest at least once in a while so you can be confident that your result is
correct.
4. Testing too many elements at once
If you are testing too many different elements at once, it is impossible to determine where the test’s
success or failure comes from. You can always go back and test different elements later as a follow-up test
or consider MVTs for more granular reporting to better determine high/low-performing components of your
test.
Flexible support options
When using an AB testing tool you need to make a decision about which solution is right for your business. Do
you take the technology/platform only or do you need some guidance and support? Our approach is flexible and
bespoke. You might be considering conversion rate
optimisation for the first time, maybe you have a dedicated internal CRO function, or perhaps you
can deal with the AB testing basics but want help with something more complex.
Whatever your requirements, our comprehensive range of support options can be tailored to your specific
needs. From a fully-managed service to a platform only SaaS model – and all points in between. Simply choose
the level of support that works for you.
View
Service Options
