Champion/Challenger Whole Email test in Engagement Program sent only Champion despite 50/50 setup

Question

I’m troubleshooting a Champion/Challenger Whole Email test in a Marketo Engagement Program and would appreciate a sanity check before opening a support case.

We have an Engagement Program with individual email assets inserted directly into the stream. Email 1 had a Whole Email Champion/Challenger test configured at 50/50.

The stream cast successfully on September 10 at 10:05 a.m. EDT and sent Email 1 to 624 people. However, the Champion/Challenger testing dashboard shows 0 sends for both Champion and Challenger, and I have not been able to identify any recipients with Test Variant A or Test Variant B activity.

I’ve verified:

• The tested Email 1 is inserted directly into the Engagement Program stream, not inside a nested program.
• Email 1 was active in the stream.
• The test type was Whole Emails.
• Distribution was 50% Champion / 50% Challenger.
• Challenger 1 was created and approved.
• The Whole Emails test itself was approved before the cast.
• Audit Trail shows:

7:55:41 a.m. EDT: Whole Emails Test Group created
7:56:02 a.m. EDT: Challenger 1 created
7:56:13 a.m. EDT: Whole Emails Test Group approved
10:05 a.m. EDT: Engagement Program cast
• The Engagement Program shows 624 sends for Email 1.

One oddity: the Audit Trail Change Details for creation of the Whole Emails test group says “Email Subject Line Test Created,” even though the test UI clearly shows a Whole Emails test.

Based on the Community documentation I’ve found, an approved Champion/Challenger test attached to an email directly in an Engagement Program stream should distribute the variants during the cast.

Is there another requirement or known limitation I’m missing? Has anyone seen a Whole Email Champion/Challenger test get bypassed like this in an Engagement Program?

I’m also setting up the same test on Email 2 for our next cast and would like to understand this before it runs.

BlaneMcMichen-1 · Answer

Supported setupCreate a regular email asset, attach aChampion/Challenger – Whole Emailstest to it, and add that email directly to the Engagement Program stream.Donotcreate an Email Program with the classic A/B test and try to nest it in the Engagement Program.StepsCreate or select the champion emailGo toMarketing Activities.Create or select the email that will be your current/primary version.Approve the email if necessary.Create the Champion/Challenger testSelect the email.ChooseEmail Actions > New Test.In the test editor, selectWhole Emailsas the test type.The original email is theChampion.Select or create the alternate email(s) as theChallenger(s).Set the percentage of recipients who should receive the challenger version. The remaining audience receives the champion version.Configure the testClickNext.Select the winner criteria and configure report alerts as needed.For an ongoing Engagement Program, the variants continue to be distributed according to the configured split until you declare a winner.Approve the testApprove the challenger email first.Then right-click the email test and selectApprove Test.Add the tested email directly to the streamOpen the Engagement Program and go toStreams.ChooseAdd Content > Email, then select the tested email.Alternatively, right-click the email and chooseAdd to [Engagement Program], then select the target stream.Ensure the email appears directly in the stream—not inside a nested Email Program.Activate the stream contentHover over the email in the stream.Click the gear icon and chooseActivate.Run a test cast with a unique test person before enabling production sends.Declare the winnerAfter collecting enough data, right-click the test and selectDeclare Champion.Choose the winning version and set the effective date.Until that date, Marketo continues sending the existing champion/challenger split; afterward, future recipients receive the declared champion.

Supported setup

Steps

Sign up

Login with SSO

Login to the community

Login with SSO