A/B testing Questions | Community
Skip to main content
Ketki_Sharma
Ketki_Sharma
Level 2
August 21, 2019
Solved

A/B testing Questions

  • August 21, 2019
  • 1 reply
  • 3741 views

1. I am planning to setup an A/B test to test click rates by day of the week. So in that case Date/Time-based A/B test would be appropriate? I have a question from this page which is one of the final step of setting up the test - Schedule the A/B Test - Marketo Docs - Product Documentation .

It calls the two variations of experiment as test and winner. What is winner in this case (since we are still setting up the test, we don't have a winner). For example, we are currently sending our campaigns on Wednesday, but we have a hypothesis that Friday might be better. So we are planning to test wednesday vs friday. Which day would test and winner in this case?

2. It says on this page - Define the A/B Test Winner Criteria - Marketo Docs - Product Documentation that "Once the A/B test is over, Marketo can automatically send the winning email at the scheduled time, or you can review the results and decide which email goes out when." How do I stop marketo from sending the winner variation automatically? I do not want to send the winner variation to the rest of my recipient list (which did not take part in A/B test)?

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Josh_Pickles

Hi Ketki,

If you're using the AB testing feature, you can drag the Test Sample Size slider to 100%, meaning that 50% run through day 1 and 50% through day 2. 

If you set up the slider this way, you technically never have a winner as defined by the AB testing feature. 

If you're looking for more insight on AB testing, I really recommend watching the Making the Most out of your A/B Testing recording of Jessica Kao at this year's Adobe Summit -> Adobe Summit 2019—The Digital Experience Conference | March 24–28, 2019 

1 reply

Josh_Pickles
Josh_PicklesAccepted solution
Level 5
August 22, 2019

Hi Ketki,

If you're using the AB testing feature, you can drag the Test Sample Size slider to 100%, meaning that 50% run through day 1 and 50% through day 2. 

If you set up the slider this way, you technically never have a winner as defined by the AB testing feature. 

If you're looking for more insight on AB testing, I really recommend watching the Making the Most out of your A/B Testing recording of Jessica Kao at this year's Adobe Summit -> Adobe Summit 2019—The Digital Experience Conference | March 24–28, 2019 

    Ketki_Sharma
    Ketki_SharmaAuthor
    Level 2
    August 29, 2019

    Thank you! We set up the test as you recommended.

      Terms & ConditionsAccessibility statement

      Loading footer...