A/B testing is at least 100 years old and was first used in agriculture and clinical trials before it was adopted by marketing companies. Today it is most commonly associated with the products of digital technology.

In digital marketing A/B testing is used to compare two versions of a website, a webpage or an app to assess which version performs better. It is a trial that often measures results by consumer feedback although it can be used in far more limited groups of respondents.

The marketer takes a web page or an app screen and makes a modified version of it. Sometimes these modifications can be very minor and sometimes the modified version could be very different indeed. Half of the audience is shown the control, which is the original, and the other half is shown the variation. The engagement and experience of the two groups is analysed using a statistical engine and the testers can then measure whether the change resulted in a positive, negative or neutral effect. 

Most commonly the control is an existing, active page but sometimes A/B testing is carried out using two variants of a prototype that is not yet live.

Apart from enabling the optimisation of the user experience, this method of testing enables developers to learn why certain elements have a particular impact. It is also useful for disproving assumptions that developers have made and can now adjust. A/B testing does more than answer a single question because it can be employed repeatedly to improve experiences and conversion rates over time.

Search Engine Considerations

This method of testing meets with the approval of Google and other search engines. In some circumstances it can be abused, however, for example as a means of cloaking. Google has therefore compiled a list of best practices for testing. These include avoiding the abuse of visitor segmentation to show different content to Googlebot based on IP addresses; coding the test so that Googlebot can understand why there are different versions of a page; using special redirects reserved for testing; and running the test for as short a time as possible so as not to be seen as deceiving the search engines.