No winner for A/B Test | Community
Skip to main content
A
Anonymous
January 14, 2015
Solved

No winner for A/B Test

  • January 14, 2015
  • 6 replies
  • 2718 views
Hello,

For my first A/B test, I decided to send winning email automatically after a certain amount of time (24 hours). What happens if there is no winner? My critieria was click to open % and my test population may have been too small.

If there is no winner, I assume Marketo will not send to the rest. What do I do to send the rest of the list the email?

Thanks for all your help!!
 
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Tara_Petre
In the past, I've found that the first email in an A/B test was sent to the remaining recipients in the case of a tie.

6 replies

Dory_Viscoglio
Dory_Viscoglio
Level 10
January 14, 2015
Hi Erica, I wouldn't assume that Marketo won't arbitrarily send one as the winner based on previous experiences. If it's possible, I'd wait until your test is scheduled to end, and if it doesn't send a winner you can set up a smart campaign to the same audience except add an additional AND filter that says "not was sent email: email x or email y". This way, you won't send to people who already received either email in the test.

But like I said, I think it will most likely send one of your emails as the winner. 
Tara_Petre
Tara_PetreAccepted solution
Level 4
January 14, 2015
In the past, I've found that the first email in an A/B test was sent to the remaining recipients in the case of a tie.
    Kristen_Malkov1
    Kristen_Malkov1
    Level 7
    January 14, 2015
    Agreed, I've had the same experience as Tara but only a handful of times. Unless you're only changing something SUPER minor, the winner should be sent. Although I would try and set up future tests to run a bit longer than 24 hrs. Perhaps 48-72 if you can! People really do tend to open emails after a few days (depending on your businesses and send dates). 

     
    Josh_Hill13
    Josh_Hill13
    Level 10
    January 14, 2015
    Agree with above, you must let it wait 48-72 hours as Kristen says before you have actionable data.

    Also, try to have at least 300 people in the test sample. Even though 30 would be the rule of thumb, you likely won't get useful data from a 30 person sample. I posted a statistical significance calculator elsewhere, but you can do a search for one online.
    Scott_McKeighe2
    Scott_McKeighe2
    Level 4
    January 14, 2015
    For reference, here's a great significance calculator for your toolbox:
    getdatadriven.com/ab-significance-test
    A
    Anonymous
    January 15, 2015
    Thank you all for your insight. It is all very helpful. In the event of a tie, the first test in the A/B test is declared the winner. Thank you.
    Terms & ConditionsAccessibility statement

    Loading footer...