Can A/B test with 3 emails? | Community
Skip to main content
May 27, 2014
Solved

Can A/B test with 3 emails?

  • May 27, 2014
  • 7 replies
  • 2323 views
Can you A/B test with 3 emails?
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Veronica_Holme4
Hi guys

You can absolutely test more than 2 whole emails, but you need to be using the Email Program type.

Set up your whole email test, and select your first two emails (you need to click "Add" to add them into the test). Once you have two set up, Marketo will give you the option to add a third, and a fourth, etc.

Be careful that you are sending to a large enough sample size to warrant multivariate testing, and be VERY careful to only change a single aspect of your email (for instance, placement of call to action on the page). Too many variables does as others have noted make it very difficult to ascertain what was actually the successful factor in the test.

Have a look at this article for more info: https://community.marketo.com/MarketoArticle?id=kA050000000LC9XCAW

7 replies

May 27, 2014
Hi Karis,
the basic function of A/B tests is just made for 2 emails. (Otherwise it would be called multivariate testing)

Best regards

Steffen
May 27, 2014
Thanks, Steffen. Do you by chance know if marketo has a multivariate testing function? 
May 27, 2014
Hi Karis,
I could be wrong, but as far as I know: No with the standard function.

Best regards

Steffen
Level 9
May 27, 2014
What are you trying to test? If you are using the A/B test feature I believe you should be able to test up to 10 of one variant (subject line, time, from address) but if it's whole emails you can just test 2.

You could probably go about this the old school way and create whole emails and then in the send email flow step you'd say if random sample = (whatever %), send (whatever email).

Hope this helps!
Kim
May 27, 2014
Kim is of cource correct on that (more variants on subject line, time, from adress) and 10 is correct!! Thought this would be about whole emails, sorry for that.

Best regards

Steffen
Veronica_Holme4
Veronica_Holme4Accepted solution
Level 9
May 28, 2014
Hi guys

You can absolutely test more than 2 whole emails, but you need to be using the Email Program type.

Set up your whole email test, and select your first two emails (you need to click "Add" to add them into the test). Once you have two set up, Marketo will give you the option to add a third, and a fourth, etc.

Be careful that you are sending to a large enough sample size to warrant multivariate testing, and be VERY careful to only change a single aspect of your email (for instance, placement of call to action on the page). Too many variables does as others have noted make it very difficult to ascertain what was actually the successful factor in the test.

Have a look at this article for more info: https://community.marketo.com/MarketoArticle?id=kA050000000LC9XCAW
May 28, 2014
Thanks all for your help!