Two A/B tests on one page - audience split question

Question

Hi! I've been asked to create two A/B tests to run simultaneously on the same page, both with an A/B/C 33% random split of All Visitors.

The two tests modify different page elements with no overlap, so I'm hoping there wouldn't be any collision. (Right?)

But I haven't been able to find a clear answer on whether the 33% traffic split is shared between the two tests. Does test 1 create a random audience split, and test 2 create a re-randomised and different audience split? Or does everyone who sees test 1 variant A, also see test 2 variant A?

Ideally I'd like the results from the two tests to be uncontaminated by one another.

(I considered approaching this as a multivariate test but we are not interested in how the two tested elements affect each other. We want to consider them as 2 distinct tests.)

Would really appreciate some clarification here, thank you!

BLX9 · Accepted Answer

Yes, you are correct that the two tests will generate their own splits for their respective variants. There will be cross over / cross contamination between the two tests. Test 1 Variant A could see Test 2 Variant A/B/C.

If you want the traffic to be mutually exclusive, where there is no cross over, then you can display Test 1 to half of your page traffic and Test 2 to the other half.

So of your 100% of page traffic... 50% Test 1 A/B/C (16.67% of total page traffic per variant) and 50% Test 2 A/B/C. It's important to factor this into your expected test duration.

Check out this page for details on mutually exclusive activities on how to configure: https://experienceleague.adobe.com/en/docs/target/using/audiences/visitor-profiles/use-profile-scripts-to-test-mutually-exclusive-activities

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded