A/B Testing

Hi Kyle,

I think you need to change your Random Sample flow steps a bit. Your choices should all be Random Sample percentages totalling 100, and your default should be "Do Nothing". Something like this:

If Random Sample is 12 Send Email #1
If Random Sample is 13 Send Email #2
If Random Sample is 12 Send Email #3
If Random Sample is 13 Send Email #4
If Random Sample is 12 Send Email #5
If Random Sample is 13 Send Email #6
If Random Sample is 12 Send Email #7
If Random Sample is 13 Send Email #8
Default: Do Nothing

Hi Drew,

I set up the random sample as you suggested above. Unfortunately, the program only went though the 8 emails and then stopped. It didn't restart at Email #1 and continue to deploy the campaign. 

Kyle

Hi Josh,

My campaign consistes for 8 different emails. I am testing the campaign on my seed list, which is 15 email addresses (this is where emails 9-15 come in).

What I had in mind when I set up the random sample choice in the flow step was that the campaign would send every email address a different email (from email #1- email #8). So, once it send the first 8 email addresses in the campaign email #1-email #8, then it would start again and the 9th email address in the campaign would receive email #1, then email address 10 would receive email #2, and so on.

When I originally set up my campaign with:

If Random Sample is 12 Send #1
If Random Sample is 13 Send #2
If Random Sample is 12 Send #3
If Random Sample is 13 Send #4
If Random Sample is 12 Send #5
If Random Sample is 13 Send #6
If Random Sample is 12 Send #7
Default: Send email #8

I thought that was the proper way to do it. And when I sent the campaign to my seed list, the campaign would properly send emails 1-8, but the rest of the email addresses afterwards would only receive the default email #8.

I am a little confused about your suggestion about 8 seperate randomly generated lists? Can you help explain

Hi Kyle,

If you setup the flow I recommended with the 8 different emails as random samples and the default as "Do Nothing", it should properly distribute the 8 different emails over the entire distribution list. However, the Random Sample (as far as I'm aware) doesn't guarantee a linear process like you're imagining, where contact #9 receives email #1 and restarts the order. The Random Sample only guarantees that the emails will be distributed in the proportions you designate.

Drew

Drew's right, you can't specify who gets what, that defeats the purpose fo the sample being random. If instead you'd like it to be precise, then create 8 lists, add the people to get email one to list one, and so on. Then member of list=list one, send email one, etc. And not use random sample at all.

Otherwise it's anybody's game, anyone who qualifies has the ability to recieve any email in the sample.