Expand my Community achievements bar.

Join us for an upcoming in-person Adobe Target Skill Builders event ~~> We're hosting these live learning opportunities to equip you with the knowledge and skills to leverage Target successfully. Learn more to see if we'll be coming to a city near you!

How to address sample size mismatch in Test

Avatar

Level 3

Traffic split 50/50%, but while analyzing data it shows sample size mismatch how to address sample size mismatch.

We have restarted the test by duplicating original test and then activating new one and paused the old test.

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

2 Replies

Avatar

Community Advisor

Dear @Ramya_SR ,

Guess we already had this discussion, however, let me ask you a few questions.

Where are you seeing this 'sample size mismatch' error/insight? Are you using some other tool to analyze this? If yes, kindly let me know.

Besides, check out this discussion as well.

Thank You, Pratheep Arun Raj B (Arun) | NextRow DigitalTerryn Winter Analytics

Avatar

Level 3

Hi Pratheep,

 

We used cookie drop for target setup. Only Adobe Target was assigned for reporting. This target cookie was tagged to analyse data in adobe analytics. So when calculating confidence using AB test guide , it alerted SRM.

 

 

Although our intention might have been for the test to have a 50/50 split, if the actual split turned out to be different, like 45/55 or 40/60, could we still extract valuable insights to inform business decisions? Is it valid to disregard results just because there's uneven distribution, especially if there's sufficient traffic experiencing each variant?

 

Your insights on this topic is highly appreciated

 

Thanks

Ramya