What is ABn testing?
ABn testing is a type of testing where 2 or more variations of a certain element on a web page are compared
with each other to understand which variation will deliver the goal of an experiment, or prove the
hypothesis.
The ‘n’ in ABn testing refers to the multiple variations in the test. By contrast AB
testing is a simpler test, pitching just one variation against another.
How ABn testing works
Traditionally, traffic is split in equal proportion between the experiments, for example if you have 4
experiments each experiment will have 25% of the traffic.
Traffic is also divided randomly amongst each variation, to make the sample distribution arbitrary.
What can you use ABn testing for?
Some examples and ideas of what you can use ABn testing for are shown below:
- Hero images and promo banners
- Copy changes
- CTA buttons
- Price messaging
- Icons, signposting
- Various exit intent messages
- Hiding elements on the page e.g. fields in the form that are not being used
- Navigation
- Imagery on the page
- Different CSS styles
- New sales funnels
Considerations when ABn testing
Below are some key considerations and things to look out for when using ABn testing:
- ABn testing is great to be able to test more than one variation of a page or element. You should make
sure that the test page has enough traffic to divide between each of the variations and that the
conversion point is also achievable for the sample size otherwise the test will have to run for a long
period of time in order to reach a high probability and statistical significance
- Look out for a number of transformations that will be made as part of the ABn test, if it is a radical
re-style of the page it may be more efficient and reliable to build and host the new pages on a separate
URL and set up a split test
- Sometimes, the changes may be simple but the you may want to understand which specific changes exactly
are driving the lift/drop in conversions, in this case multivariate
testing (MVT) is likely to be more appropriate
- Look at the version of the test that did not perform well. Understanding what causes a particular
experiment to under-perform can help to evaluate the hypothesis and also to come up with additional
tests, not specifically generated from assumptions only, but also learnings from phase 1 of the results
- When multiple variants are tested in the first page, it may be worth it to rerun an MVT of the winning
version to understand what particular elements on the test helped to drive the increase in performance
ABn testing with Webtrends
Optimize
Webtrends Optimize can run all of the examples and ideas provided above when ABn testing. We can also provide
additional support and guidance with the strategy and process of deciding what type of tests best support
the hypotheses and outcome you are trying to achieve, for example AB, ABn, MVT, Split tests, Server-side testing etc. All are available in the Webtrends Optimize
platform as standard.
