A/B Testing Data Questions | Community
Skip to main content
Jason_Scott
Level 4
July 7, 2014
Question

A/B Testing Data Questions

  • July 7, 2014
  • 2 replies
  • 1064 views

It seems when an a/b/n test is excuted all the "A" versions are sent then all the "B" and so on.  Can anyone verify this?
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

2 replies

Josh_Hill13
Level 10
July 7, 2014
Yes, that is how it should be.

You should select the A version to send during the setup process. The B versions are also set.

Are you experiencing something else?
Jason_Scott
Level 4
July 7, 2014
Thanks for the reply.

That is what I'm seeing however, this leads to some poor data upon execution.  Deliverability starts to decline as an email is sent and therefore the "C" or "D" and so on versions gets poorer and poorer.  Every test we’ve run “A” has the highest deliverability followed by “B” then “C” and so on.

I believe Marketo is also assigning everyone within the campaign to a test group then deduping.  Once the email is executed “A” sends then “B” however “B” excludes any dup that was sent “A.” So, “B” test group is now a lower number of samples.  Then if there’s a “C” test group it’s even lower since any dup sent “A” or “B” is excluded.