Control groups in campaigns

[rohitcts]

which type of campaigns have greater need for a control group.

a. an email campaign with links to the e-commerce site.

b.a direct mail campaign with coupon code

c.a banner on the website advertising a in-store sale

answers with explainations expected.

[Milan_Vucetic · Accepted solution]

Hi @rohitcts,

basically control group should be used in almost every campaign. They enable us to show you whether or not a message drove an increase (or decrease) in user engagement, revenue, or conversion. Only two cases when you should avoid control group is:

• Urgent Notifications – There are certain notifications that you want to make sure get to every user. For example, if you’ve discovered a bug and need users to update to a new version of your app, that is a situation where you shouldn’t use a control group.
• Very Small Audiences – When you have an audience that is only a few hundred people, then the control group won’t be big enough to be statistically significant. In other words, you won’t be able to trust that the control group will accurately show you how the audience would behave if they didn’t receive a message.

Regards,
Milan

•  

[abhshett]

C. A banner on the website advertising a in-store sale

[anupk2111]

Agree with @Milan_Vucetic 

In addition, I imagine you would like to use a control group in all campaigns that have measurable KPIs like revenue, signups etc. So that you are able to compare and find out how effective has the campaign been. Also, it would be a good idea to keep the control group consistent across all communication belonging to a particular campaign ( For example - Summer Collection sales promotion email1 and reminder email2.). This can give you a better idea of how the control group is behaving.

For testing the validity of content variations ( like banners/ subject lines), it would rather be more helpful to use AB tests instead of control group.