Highlighted

A/B Test Visit inflation - A4T reporting

markm49192534

15-11-2018

Hello,

I have an A/B test where there is a 90%/10% split in traffic (unique visitors are respecting this split), but visits for the groups are coming out as 85%/15%. Is there a reason why we're seeing this higher concentration of visits in one group? 

Also, is there any chance that Unique visitors could be reporting low in either group?

Replies

Highlighted

mikewebguy

Employee

15-11-2018

markm56939586​,

Here is how I would think about it:

My traffic by visitor not visit is being split 90/10 after that there is no restriction on the number of visits each visitor has so in essence:

90% of visitors generated 85% of total visits.

&

10% of visitors generated 15% of total visits.


If you let the activity run for longer and wait for a lot more visitors I bet these numbers will gravitate closer and closer towards 90% 10% distribution just like the visitors. Hope this helps!

Mihnea Docea  | Technical Support Rep, Customer Care | Customer & Employee Experience | Adobe | (:: 1 (800) 497-0335

Highlighted

markm49192534

15-11-2018

Thanks for the response. I realize it's return visits, but there isn't a reason for one group to have more return visits than another. There are millions of visitors in the test. It seems like an anomaly for there to be a larger amount of visits per unique in one group over the other.

Highlighted

mikewebguy

Employee

15-11-2018

markm56939586​,

Since we are both on the same page about return visitors. Help me understand what numbers you would expect to see for visits percentage when looking at ExpA vs ExpB curently while you have 1:9 visitor ratio between the experiences the visit ratio is a bit lower 1:5.6

For Visitors:

ExpA: 90% Visitors

ExpB: 10% Visitors

The ratio is 1:9 (So for every one visitor in ExpB we have 9 visitors in ExpA)

For Visits:

ExpA: 85 Visits

ExpB: 15 Visits

The ratio is 1:5.6 (So for ever one visit in ExpB we have 5.6 visits in ExpA)