Guidance on Launching A/B Test | Community
Skip to main content
A
Anonymous
July 29, 2016
Solved

Guidance on Launching A/B Test

  • July 29, 2016
  • 1 reply
  • 3190 views

Hello,

I am currently working to launch an A/B test with nine different email variations. I want to use the out of the box functionality Marketo has for this and believe I know how to set this up. HOWEVER, my contacts are contained within a Smart List. How do I set the A/B test up to to pull from the Smart List I have?

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Grégoire_Miche2

Hi David,

Are you using an email program or a champion challenger A/B test ? If it's anemail program, just run it on your smart list. If a champion challenger, create a  batch smart campaign with the smart list as a target and use an send email flow step, using the test group as the email.

In any case, just send the email as usual, and let Marketo randomly assign each lead to a variant.

Now I would be a little suspicious about the relevance of a test with 9 variants. If you end up sending each variant to less than a few thousands of people, the statistical relevance of the result will be questionable.

Also, avoid testing multiple criteria on the same time, you'll have a harder time making sens of the results.

-Greg

1 reply

Grégoire_Miche2
Grégoire_Miche2Accepted solution
Level 10
July 29, 2016

Hi David,

Are you using an email program or a champion challenger A/B test ? If it's anemail program, just run it on your smart list. If a champion challenger, create a  batch smart campaign with the smart list as a target and use an send email flow step, using the test group as the email.

In any case, just send the email as usual, and let Marketo randomly assign each lead to a variant.

Now I would be a little suspicious about the relevance of a test with 9 variants. If you end up sending each variant to less than a few thousands of people, the statistical relevance of the result will be questionable.

Also, avoid testing multiple criteria on the same time, you'll have a harder time making sens of the results.

-Greg

A
Anonymous
July 29, 2016

Thanks, Greg. I'm working to set this up as a champion challenger A/B test (The option used by right clicking on the email and selecting add test). So once the A/B test is setup, and the variants defined, I just send the contacts the email in a Smart Campaign and Marketo will do the allocation? Just want to make sure I have this straight.

Grégoire_Miche2
Grégoire_Miche2
Level 10
July 29, 2016

Hi again David,

yes this is exactly it

-Greg

Terms & ConditionsAccessibility statement

Loading footer...