A/B Testing whole emails within engagement program | Community
Skip to main content
Diana_DeMallie
Level 2
December 13, 2019
Question

A/B Testing whole emails within engagement program

  • December 13, 2019
  • 2 replies
  • 5122 views

I'm trying to A/B test two different versions of an email within a nurture engagement program.

I've set it up using the "If Random Sample = 50" setup in the flow step of the batch send campaign. 

Would one email be in the Choice 1 option, with the other email as the default choice? (attached).

Or, is the way to do this to use no default choice, and instead, use:

- If Choice 1 = Random Sample 50

Email: version 1

- If Choice 2 = Random Sample 50

Email Version 2

Default choice is empty

Thanks!

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

2 replies

Level 2
December 16, 2019

@Diana DeMallie‌ - Both are fine to use. Also, seems that the attachment is incorrect.

SanfordWhiteman
Level 10
December 16, 2019

Actually that's not so, you should use a (non-empty) Default Choice whenever using Random Sample.

Not every batch is evenly divisible. And even when the count is divisible on a mathematical level, Random Sample does not behave like a round-robin algorithm. 

Diana_DeMallie
Level 2
December 16, 2019

Got it - thanks!

After this launches, will a regular email performance report be the best way to view and compare the results between the two emails that are split with the random sample? 

Tracey_Bartz1
Level 4
December 16, 2019

You may also want to consider using the champion/challenger test functionality instead: Email Tests - Champion/Challenger - Marketo Docs - Product Documentation