Expand my Community achievements bar.

Applications for the 2024 Adobe Target Community Mentorship Program are open! Click to the right to learn more about participating as either an Aspirant, to professionally level up with a new Certification, or as a Mentor, to share your Adobe Target expertise and inspire through your leadership! Submit your application today.
SOLVED

Required amount of traffic for AB test percentages to match up to those defined in activity

Avatar

Level 5

Hi everyone!

We've just deployed an A/B test with a simple 50/50 split.

The test has been running for 3 days and the current split is 66% control, 34% test for 222 visitors (we're using Analytics for Target as the reporting source). Yesterday, it was an 80/20 split between the control and test variants.

The test itself deploys a client side redirect if the customer has a specific cookie value (we're not using the built in redirect to page option in Target as we need to modify the cookie value before applying the redirect). The audience is based on customers hitting the site with a specific query parameter value.

There haven't been any changes since activation (i.e. pausing and reactivated, changes to audience, etc.) so I'm just wondering if there's a ballpark figure where we can expect the numbers to even out as required.

Any information would be greatly appreciated.

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

1 Accepted Solution

Avatar

Correct answer by
Administrator

[8.3.22 Adobe Target Community Q&A Coffee Break Answer provided by @rbooth115

----------------------------

Hi @philipk92120636 ! Did you use the Target Sample Size Calculator to estimate how long this test needs to run before it reaches significance? I recommend that you don't analyze the results of an A/B tests until it has reached significance (ie: ran for X number of days as defined by the calculator in order to reach significance). Until then, the data might not make sense. 

Further, please note that a 50/50 traffic split does not mean that every other visitor is sent to the control experience, rather it is a flip of the coin every time, thus it will take a certain amount of site traffic and time for this to even out.

----------------------------

Check out the entire 8.3.22 Adobe Target Community Q&A Coffee Break discussion thread for more insights on related topics  

View solution in original post

1 Reply

Avatar

Correct answer by
Administrator

[8.3.22 Adobe Target Community Q&A Coffee Break Answer provided by @rbooth115

----------------------------

Hi @philipk92120636 ! Did you use the Target Sample Size Calculator to estimate how long this test needs to run before it reaches significance? I recommend that you don't analyze the results of an A/B tests until it has reached significance (ie: ran for X number of days as defined by the calculator in order to reach significance). Until then, the data might not make sense. 

Further, please note that a 50/50 traffic split does not mean that every other visitor is sent to the control experience, rather it is a flip of the coin every time, thus it will take a certain amount of site traffic and time for this to even out.

----------------------------

Check out the entire 8.3.22 Adobe Target Community Q&A Coffee Break discussion thread for more insights on related topics