Skip to main content
Mar 31, 2015 3:57 AM UTC
Solved

AB Test Whole Emails in Engagement Program

We have an active Engagement Program turned on with only one Stream that has 3 emails in it. We would like to AB test two whole emails with a random distribution of records for the first email.

Any recommendations on how to set this up?

Thanks in advance!
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Accepted solution
Mar 31, 2015 3:10 PM UTC
Please excuse the blackouts :)




9 replies

Mar 31, 2015 2:44 PM UTC
I'd recommend using programs within your engagement stream and a smart campaign to define your members and the A/B flow steps. Programs open up another world of fun you wouldn't really be able to do otherwise.

How I structured my A/B testing is by cloning the email and making my changes on the clone. Using the native A/B test feature within Engagement programs has caused problems. There's disucssions on this throughout the community.

Within your flow step of the smart campaign, use the random sample restrain to define your split.

After your program is all set place it into the nurture and when prompted, select your Smart Campaign you just built to ensure your email gets pulled into the native engagement program send.

I have screenshots if you need.
Translate
Mar 31, 2015 2:51 PM UTC
Thanks for the qick response Tim! Yes I want to avoid using the Champion/Challenger by all means! Can you share screen shots of what you built? Much appreciated!
Translate
Accepted solution
Mar 31, 2015 3:10 PM UTC
Please excuse the blackouts :)




Translate
Apr 3, 2015 12:38 AM UTC
Thanks so much for sending the screen shots Tim! Did you setup the smart campaign in your program as a bi-weekly batch? How does the campaign/stream know when to fire the email?
Translate
Apr 3, 2015 12:52 AM UTC
You can set your smart campaign like any other. No need to schedule things out. Drop the program in your stream. It will prompt you to select the campaign you just made. Use the streams program cadence as the sending mechanism. The engagement program naturally u sweat and to send on the days and times you set.
Translate
Apr 3, 2015 1:18 AM UTC
Ah Gotcha! Thanks...was curious how that work but totally making sense now!
Translate
Apr 3, 2015 8:28 PM UTC
Sorry to bug you again Tim! What are the pros and cons of using a Program versus Champion/Challenger for AB Testing?
Translate
Apr 3, 2015 8:49 PM UTC
Programs can use both trigger and batch campaigns to A/B test while using champion challenger restricts you to just using trigger campaigns. Marketo says they work in engagement programs, but the documentation on this is conflicting. Basically there is no pro or con it's just you can't use champion challenger in all situations. Wish this wasn't so, I like the idea of champion challengers.
Translate
Apr 4, 2015 3:00 AM UTC
Thanks Tim!
Translate