Skip to main content
A
Anonymous
July 14, 2014
Question

A/B Test in Engagement Program not sending Challengers

  • July 14, 2014
  • 14 replies
  • 2908 views
I have an A/B champion test on subject lines set up for one of my email programs inside an engagement program. 

We set it live this weekend where those who fill out a form are triggered to enter the stream. My cadence is set for every day and I've recieved over 40 people this weekend to enter the stream.

So far my results are showing that the program is only sending the champion email and no challengers. I have not declared a winner yet.

Any reason why this may be happening?


This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

14 replies

Show previous replies
A
Anonymous
July 23, 2014
I'm running into the same problem. All tests approved, and I'm receiving the emails to give me an update, but none of the challengers are being sent. Is there something that Support can do to fix this?

It would be very helpful to be able to A/B test within engagment programs.
Dory_Viscoglio
Dory_Viscoglio
Level 10
July 23, 2014
We also had a champion/challenger test that didn't go out (properly approved) as scheduled on July 15. Cheryl, can you confirm if this has been fixed? 
A
Anonymous
July 23, 2014
My champion/challenger has been sending out correctly. From my understanding you must place the direct email in the stream and it cannot be within a program. 

I also have no cadence set for this champion.challenger as I also hear it must be triggered instead of batch sent. 

If the above meets your campaign setup and you still  have an issue, I would open a ticket with support. I have yet to hear on a final OK this has been fixed. 
A
Anonymous
December 12, 2015

I have the same issue you had with Champion Challenger. It seems you can't use a Program within the Engagement stream. You have to use the individual emails in order to use the Champion Challenger functionality. It didn't work initially but then I tested it with just the email itself in the stream and it worked. But nowhere does it say that it wouldn't work in the documentation. I opened a ticket with Support.

A
Anonymous
December 12, 2015

Hi Anthony,

If you're using programs in engagement streams I can only imagine the complexity to properly champion challenger for Marketo. It would be wonderful to have, but hard to build out for them.

Looks like we will need to do our reporting outside the "out of box" functionality of champion challenger.

Im always in favor for more robust documentation. Marketo cant do it all though right?

Post your support update, be curious to hear what they say. 

A
Anonymous
December 13, 2015

Well it seems pretty straightforward. I created a Smart Campaign that sends the email within the program as instructed in the product docs when you want to use Programs instead of individual emails in Engagement Programs. When I export the report of the Champion/Challenger it shows that all the emails were sent for the Champion. It literally skipped the challengers. I would think in the back end programming of Marketo they could easily fix this with some type of command in the code to split the list into the various percentages as specified in the Champion Challenger.

I guess another alternative if I wanted to use Programs within Engagement Programs would be in the Flow to do a Random sample 50 percent and have it send the two emails. Reporting-wise it would be more manual but I would see the two subject lines in the report. Then I could track how it is performing an manually remove one of the emails. Not as easy as just "declaring a winner" but a workaround nonetheless.

I think it was an oversight on their part when they created Champion/Challenger. The documentation says clearly: "Champion/Challenger emails only work with trigger campaigns and engagement programs. For batch mailings, use the email program's A/B testing functionality."

Terms & ConditionsAccessibility statement

Loading footer...