Control groups in campaigns | Community
Skip to main content
R
rohitcts
July 14, 2020
Solved

Control groups in campaigns

  • July 14, 2020
  • 3 replies
  • 2455 views

which type of campaigns have greater need for a control group.

a. an email campaign with links to the e-commerce site.

b.a direct mail campaign with coupon code

c.a banner on the website advertising a in-store sale

 

answers with explainations expected.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Milan_Vucetic

Hi @rohitcts,

basically control group should be used in almost every campaign. They enable us to show you whether or not a message drove an increase (or decrease) in user engagement, revenue, or conversion. Only two cases when you should avoid control group is:

  • Urgent Notifications – There are certain notifications that you want to make sure get to every user. For example, if you’ve discovered a bug and need users to update to a new version of your app, that is a situation where you shouldn’t use a control group.
  • Very Small Audiences – When you have an audience that is only a few hundred people, then the control group won’t be big enough to be statistically significant. In other words, you won’t be able to trust that the control group will accurately show you how the audience would behave if they didn’t receive a message.

Regards,
Milan

  •  

3 replies

Milan_Vucetic
Milan_VuceticAccepted solution
Level 9
July 15, 2020

Hi @rohitcts,

basically control group should be used in almost every campaign. They enable us to show you whether or not a message drove an increase (or decrease) in user engagement, revenue, or conversion. Only two cases when you should avoid control group is:

  • Urgent Notifications – There are certain notifications that you want to make sure get to every user. For example, if you’ve discovered a bug and need users to update to a new version of your app, that is a situation where you shouldn’t use a control group.
  • Very Small Audiences – When you have an audience that is only a few hundred people, then the control group won’t be big enough to be statistically significant. In other words, you won’t be able to trust that the control group will accurately show you how the audience would behave if they didn’t receive a message.

Regards,
Milan

  •  
    abhshett
    abhshett
    Level 2
    July 17, 2020

    C. A banner on the website advertising a in-store sale

    anupk2111
    anupk2111
    Level 2
    July 22, 2020

    Agree with @milan_vucetic 

     

    In addition, I imagine you would like to use a control group in all campaigns that have measurable KPIs like revenue, signups etc. So that you are able to compare and find out how effective has the campaign been. Also, it would be a good idea to keep the control group consistent across all communication belonging to a particular campaign ( For example - Summer Collection sales promotion email1 and reminder email2.). This can give you a better idea of how the control group is behaving.

     

    For testing the validity of content variations ( like banners/ subject lines), it would rather be more helpful to use AB tests instead of control group.

     

    Terms & ConditionsAccessibility statement

    Loading footer...