A good refresher for the marketing department

What is A/B Testing?

The intuition is to test two different methods or elements before landing on one choice. This is done in many industries, scientific analysis, medicinal studies, marketing departments, etc. The focus of this post is on the marketing department.

A/B testing is also like a pilot study. Before mass marketing a campaign and spending thousands, you want to determine whether the approach is right or not. Say you are launching a year-end marketing campaign and have to decide whether to offer a buy one get one free promotion or a free shipping promotion. How do you decide on one?

That’s where A/B tasting comes into play. You test both these methods on a small number of customers to observe their reactions and the return of the campaign. The one with the higher return is the one you stick with for mass marketing. This prevents a potential failure for a campaign and saves a lot of money.

The Right & Wrong

A/B test is a very simple calculation of which campaign promotion generated more revenue and performed better. You would have to try very hard to get the calculation itself wrong. So, How can this go wrong?

A campaign might fail despite showing better results compared to the alternative method during the A/B testing. The real reason for that is poor target audience selection. More often than not, campaigns fail to produce significant returns for this reason alone.

Here is an analogy;

Let’s say you are conducting an A/B test between free shipping and buy one get one free for clothing sections. For argument’s sake, let’s say the buy one get one free would have generated much better results for the year-end campaign. However, the audience selection for this campaign during the A/B test was very irrelevant. This would generate poor results for this campaign selection, and you are forced to choose the free shipping promotion. As a result, the actual marketing campaign failed to generate a significant return, which could have been a completely different story if the target selection was better for the buy one to get one free choice.

The point is, just because the A/B test shows results, you should not blindly accept it. A/B test would always be right in terms of calculations; what you should be asking is whether the target audience selection was propper or not. This way, you evaluate your options more accurately, the results would portrait much closer to the actual picture. You would be spending the campaign money on something more significant and relevant.

Next week I will be explaining how the Target audience for the A/B test can be chosen using the help of data science and software called Enhencer.