Acceptable Traffic Split Deviation in Adobe Target A/B Testing

Question

I am running an A/B test in Adobe Target with a 50/50 traffic split configuration. However, upon reviewing the data, I noticed that the actual split is 49.8% (variant) vs. 50.5% (control)—a difference of approximately 0.7%.

I would like to understand:

Is this level of deviation (~0.7%) considered normal in Adobe Target A/B tests?
Does Adobe Target enforce an exact 50/50 split, or is there an expected margin of error?
At what level of imbalance should I be concerned about test validity?
The test is still ongoing, and I want to ensure that this traffic allocation discrepancy does not impact the final results. Any official Adobe documentation or expert insights would be greatly appreciated.

Vaibhav_Mathur · Accepted Answer

Hi @shoebsh ,

Quoting few lines from the documentation, "As a result of random splitting of traffic, it can take a significant amount of initial traffic before the percentages even out. For example, if you create two experiences, the starting experience is chosen randomly. If there is little traffic, it’s possible that the percentage of visitors can be skewed toward one experience. As traffic increases, the percentages equalize."

Reference : https://experienceleague.adobe.com/en/docs/target/using/introduction/how-target-works

I don't think there is any given acceptable level of deviation in the documents, but a normal expected deviation according to my experience in less traffic pages can be around 1%-2%.

A best practice to manage this deviation can be to run an A/A test on the experiment page beforehand to analyse the traffic and the accuracy of traffic allocation on the given page.

For reference : https://experienceleague.adobe.com/en/docs/target/using/activities/abtest/aa-testing

I hope it helps.

Best regards,

Vaibhav Mathur

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded