Expand my Community achievements bar.

Applications for the 2024 Adobe Target Community Mentorship Program are open! Click to the right to learn more about participating as either an Aspirant, to professionally level up with a new Certification, or as a Mentor, to share your Adobe Target expertise and inspire through your leadership! Submit your application today.

A/B Test Visit inflation - A4T reporting

Avatar

Level 2

Hello,

I have an A/B test where there is a 90%/10% split in traffic (unique visitors are respecting this split), but visits for the groups are coming out as 85%/15%. Is there a reason why we're seeing this higher concentration of visits in one group? 

Also, is there any chance that Unique visitors could be reporting low in either group?

3 Replies

Avatar

Employee Advisor

markm56939586​,

Here is how I would think about it:

My traffic by visitor not visit is being split 90/10 after that there is no restriction on the number of visits each visitor has so in essence:

90% of visitors generated 85% of total visits.

&

10% of visitors generated 15% of total visits.


If you let the activity run for longer and wait for a lot more visitors I bet these numbers will gravitate closer and closer towards 90% 10% distribution just like the visitors. Hope this helps!

Mihnea Docea  | Technical Support Rep, Customer Care | Customer & Employee Experience | Adobe | (:: 1 (800) 497-0335

Avatar

Level 2

Thanks for the response. I realize it's return visits, but there isn't a reason for one group to have more return visits than another. There are millions of visitors in the test. It seems like an anomaly for there to be a larger amount of visits per unique in one group over the other.

Avatar

Employee Advisor

markm56939586​,

Since we are both on the same page about return visitors. Help me understand what numbers you would expect to see for visits percentage when looking at ExpA vs ExpB curently while you have 1:9 visitor ratio between the experiences the visit ratio is a bit lower 1:5.6

For Visitors:

ExpA: 90% Visitors

ExpB: 10% Visitors

The ratio is 1:9 (So for every one visitor in ExpB we have 9 visitors in ExpA)

For Visits:

ExpA: 85 Visits

ExpB: 15 Visits

The ratio is 1:5.6 (So for ever one visit in ExpB we have 5.6 visits in ExpA)