Can you 50/50 test a whole email? | Community
Skip to main content
Michaela_Morale
Michaela_Morale
Level 2
January 31, 2019
Question

Can you 50/50 test a whole email?

  • January 31, 2019
  • 4 replies
  • 8659 views

I want to set up an A/B test for an email send. I have 2 versions of the email. I want to send it 50/50 to everyone. Can this be set up through the A/B test option? I don't want the winner to be sent out since I want it to all go out at one time and then see which is possible.

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

    4 replies

    Josh_Hill13
    Josh_Hill13
    Level 10
    January 31, 2019

    Sure.

    Just set the Sample to 50/50 and there will be no winner sent. Do you mean something else?


    Probably best to use Random Sample method.

    Michaela_Morale
    Michaela_MoraleAuthor
    Level 2
    January 31, 2019

    I guess I'm unclear how to do that since the slider options are Receives Test and Receives Winner

    Valerie_Whitin2
    Valerie_Whitin2
    Level 4
    January 31, 2019

    Hi @Michaela Morales​

    You would set the test sample size to 100% for it to work that way. That means it will evenly split your test among 100% of the audience you selected. Which would in turn be a true 50/50 split. It will still make you select a time to "send the winner" but really nothing will happen other than you will get an email with the results of the test. This is how I've almost always run my tests and have never had any issues. I hope this helps answer your question

      Michaela_Morale
      Michaela_MoraleAuthor
      Level 2
      January 31, 2019

      Yes it does, thank you Valerie!

      N
      npasero
      Level 3
      February 1, 2019

      Usually in my team we used to create two different programs for making A/B Testing:

      Program 1: Sends version A

      Program 2: Sends version B

      This would be easily displayed on the reporting.

      Nithish_Bhat
      Nithish_Bhat
      Level 2
      February 1, 2019

      @Natascia Almeida​ What about the list/audience? are they same or do you split it ?

      N
      npasero
      Level 3
      February 1, 2019

      We split it

      Natalie_Kremer
      Natalie_Kremer
      Level 4
      February 7, 2019

      We do this often by just using a Smart Campaign to send the email. We determine the audience in the Smart List like we normally would, but within the Flow step, we choose Send Email and Add Choice. The criteria would be Random Sample is 50 which would send your Email A and then you'd set Email B as the default email to send. After you send you can then run your Email Performance Report to look at the different metrics and determine which performed better. I find this gives you more control than the out of the box A/B test.

      Here's an example of the Flow:

        Tiffany_Scott
        Tiffany_Scott
        Level 2
        April 23, 2020

        This was one of the best tips I've ever seen on here- thanks a lot @natalie_kremer !

         

        Would you recommend doing this/ will it even work within an engagement program that uses programs instead of emails to send an email? I fear it may be too taxing on the system if there are too many programs that would need to call for this random sample.  

         

        Tiffany Scott
        Terms & ConditionsAccessibility statement

        Loading footer...