A

Anonymous

Solved

A/B Testing - Static Lists Only?

Forum|Forum|12 years ago
October 23, 2013
8 replies
2360 views

If I'm reading the documentation correctly

The only method to conduct a 50/50 AB test is by creating 2 static lists.

Is there a method to use a smart list (or some other means)?

Thank You.
Tim

Marketo

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by

Hi Timmy,

I would add Email 2 as the default step and then I always leave the remainder of the random percentage as default. Marketo sends to the remaining percentage that didn't receive email 1. In this case 50%.

The reason is mostly convenience. One less step to add.

The other is numbers, if you have 101 contacts and you do two 50% steps, because there is still that default step I assume the system would send 50 people email 1 and 50 people email 2, and one person doesn't receive it. A lot of thinking for 1 email, but again it's easier to just use the default so why not?

A

Anonymous

FYI - AB split test and Champion Challenger test features are due out in our December release!

Static list is the best way b/c it is a set number of leads versus with a smart list, the count can change at any time.

A

Anonymous

What do we do when contacts are being added 24x7x365 ??

With a static list..... new people (potentially more important) may be ommitted...

Especially if there are Next Steps in the campaign.... Would this mean we'd be creating a new static list for every execution of a campaign?

OR how would we automatically repopulate the static lists?

A

Anonymous

Yay Champion Challenger release!!

Timmy,

Use flow step Send Email and then select "add choice" twice (if you are testing two things). Then you would put If RANDOM SAMPLE is 50 send EMAIL 1 and for the second choice if RANDOM SAMPLE is 50 send EMAIL 2.

A

Anonymous

Are you A/B testing a landing page or an email?

If it's an email:

Select the "send email" action
In the flow, add a condition to the "send email" step
Set the condition to "Random Sample" = 50
Set the first step to email A
Set the default action to email B

If you're wanting to update static lists to keep them 50/50 it would be basically the same flow steps except you would change "send email" to "add to list" and seperate your a & b lists.

A

Anonymous

Josh and Kim

Thank you for your kind replies.

Josh,
RE: 50% + Default

Please clarify, you're suggesting

Random Send for Email 1

Default Send for Email 2

QUESTION - How would Marketo know the "Default Send for Email 2?"

----------

Kim
RE 50% + 50%

If I understand correctly, you're suggesting

Random 1 (50%) + Random 2 (50%)

Wouldn't this create a condition where

25% would have double send (held as it was previously sent)
+ 25% of the list that didn't receive an email?

QUESTION - If I understand, this random 50% + random 50% = 25% that do not receive the email.

---------

It doesn't make sense that we'd create 2 static lists (I do understand AB testing is in the next version)

Thank you,

Timmy

A

Anonymous

Accepted solution

Hi Timmy,

I would add Email 2 as the default step and then I always leave the remainder of the random percentage as default. Marketo sends to the remaining percentage that didn't receive email 1. In this case 50%.

The reason is mostly convenience. One less step to add.

The other is numbers, if you have 101 contacts and you do two 50% steps, because there is still that default step I assume the system would send 50 people email 1 and 50 people email 2, and one person doesn't receive it. A lot of thinking for 1 email, but again it's easier to just use the default so why not?

A

Anonymous

Thank you Josh.

A

Anonymous

I'll follow through with the results.

Hoping that documenting success will help another community user.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded