Expand my Community achievements bar.

SOLVED

Two A/B tests on one page - audience split question

Avatar

Level 1

Hi! I've been asked to create two A/B tests to run simultaneously on the same page, both with an A/B/C 33% random split of All Visitors. 

The two tests modify different page elements with no overlap, so I'm hoping there wouldn't be any collision. (Right?)

 

But I haven't been able to find a clear answer on whether the 33% traffic split is shared between the two tests. Does test 1 create a random audience split, and test 2 create a re-randomised and different audience split? Or does everyone who sees test 1 variant A, also see test 2 variant A?

Ideally I'd like the results from the two tests to be uncontaminated by one another. 

(I considered approaching this as a multivariate test but we are not interested in how the two tested elements affect each other. We want to consider them as 2 distinct tests.)

Would really appreciate some clarification here, thank you! 

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

1 Accepted Solution

Avatar

Correct answer by
Level 2

Yes, you are correct that the two tests will generate their own splits for their respective variants. There will be cross over / cross contamination between the two tests. Test 1 Variant A could see Test 2 Variant A/B/C.

 

If you want the traffic to be mutually exclusive, where there is no cross over, then you can display Test 1 to half of your page traffic and Test 2 to the other half.

So of your 100% of page traffic... 50% Test 1 A/B/C (16.67% of total page traffic per variant) and 50% Test 2 A/B/C. It's important to factor this into your expected test duration.

 

Check out this page for details on mutually exclusive activities on how to configure: https://experienceleague.adobe.com/en/docs/target/using/audiences/visitor-profiles/use-profile-scrip...

View solution in original post

2 Replies

Avatar

Correct answer by
Level 2

Yes, you are correct that the two tests will generate their own splits for their respective variants. There will be cross over / cross contamination between the two tests. Test 1 Variant A could see Test 2 Variant A/B/C.

 

If you want the traffic to be mutually exclusive, where there is no cross over, then you can display Test 1 to half of your page traffic and Test 2 to the other half.

So of your 100% of page traffic... 50% Test 1 A/B/C (16.67% of total page traffic per variant) and 50% Test 2 A/B/C. It's important to factor this into your expected test duration.

 

Check out this page for details on mutually exclusive activities on how to configure: https://experienceleague.adobe.com/en/docs/target/using/audiences/visitor-profiles/use-profile-scrip...