Expand my Community achievements bar.

SOLVED

Incorrect traffic split on ab test

Avatar

Level 2

We ran an ab-type experiment with 50/50 manual allocation with both the control and variant versions modified.

 

setup.png

 

The control version also injects custom js/css.

We have noticed an anomalous split percentage, given that the variant has approximately double the number of users of the control.

report.png

 

We can't understand what the problem could be.

Do you have any idea?

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

1 Accepted Solution

Avatar

Correct answer by
Community Advisor

@Jack_Hala In addition to what @alexbishop mentioned, did you change the split midway through the activity or make major edits post activating the activity? Sometimes that also plays a part in this anomaly.

 

https://helpx.adobe.com/ca/target/kb/traffic_allocation_in_activity_in_target.html

View solution in original post

2 Replies

Avatar

Employee Advisor

Is it only in this activity that you've seen an unequal traffic split? Have you got Target integrated to an analytics platform where you could see what the distribution looks like? I'm just thinking if there is some sort of error occurring for users on certain browsers/devices for the Control version, for example. 

I'd be tempted to run an A/A test with no changes, just to rule out a more fundamental error with traffic allocation/implementation

Avatar

Correct answer by
Community Advisor

@Jack_Hala In addition to what @alexbishop mentioned, did you change the split midway through the activity or make major edits post activating the activity? Sometimes that also plays a part in this anomaly.

 

https://helpx.adobe.com/ca/target/kb/traffic_allocation_in_activity_in_target.html