Adobe Target AB test not splitting traffic as set

[angharadm287005]

Hi all,

We've had an AB test with 1 variant running at 50:50 traffic split for the last 4 days. However we're consistently seeing more of a 52% 48% split. We aren't using auto allocate. Is there any reason for the test not being closer to 50:50?

[Gaureshk_Kodag]

In case of auto allocation, 20% of traffic is reseverd to serve random experience and 80% is served to winner experience. Now, when will traget decide who's the winner? - The algorithm starts working after all experiences in the activity have a minimum of 1,000 visitors and 50 conversions.

In your case if you're seeing bit more traffic to experience A could be because more people are converting in exp A. Also target never split traffic in exact number that is provided in targeting settings thr is always slight difference, as it continuesly splitting traffic as it comes in.

Hope this helps.

[angharadm287005]

Thank you. 

We aren't using auto allocate, we are using a manual split. And the split has been consistently uneven since the test has been live. 

How much slight difference would be considered within a normal range?

[Gaureshk_Kodag · Accepted solution]

@angharadm287005 As far as I have worked on target, I have always seen a slight difference in the split at least for a few initial days which is like 48-52, 49-51. I think you should give it some time and see the change in traffic split.

After a week or so the difference would decrease as more traffic comes in. That is normal behavior I would say.

[trahulc]

I am seeing a slightly more variance. Almost 60:40 split instead of the 50:50. We haven't selected 'Auto allocate' feature.

[akbansal1]

There could be multiple factors affecting the traffic split among your control and challenger variants.

Firstly, total traffic in the test. If the test is in the initial stage after launch and the numbers are small, you might see an unequal split. However, as the traffic increases the split should be as you have set manually in Target. Example, for a low traffic page (100 Visitors per page) you might not see an equal split initially. The overall period for the split to converge depends on traffic, and not so much on days for which the experiment has been live.

Secondly, Target and Analytics have different engines to count visitors. Yes, if you are using Analytics for Target (aka A4T) numbers should be equal and there will be an equal split. If you are using a custom code(sending Control or Challenger information into a variable) to send traffic to Adobe Analytics, you might need to check if the code runs every time. As Target and Analytics have different engines to count visitors, it is recommended to verify Traffic split of A4T activities through Target activity performance APIs. APIs can be referred here >  Target API v1.0

Thirdly, Visitors in an A/B test will stay in their initially-assigned experiences. The traffic change impacts new entrants only. Are you sure you don't have large number of repeat visitors?