How to address sample size mismatch in Test | Community
Skip to main content
R
Ramya_SR
Level 2
August 14, 2023
Question

How to address sample size mismatch in Test

  • August 14, 2023
  • 1 reply
  • 1047 views

Traffic split 50/50%, but while analyzing data it shows sample size mismatch how to address sample size mismatch.

We have restarted the test by duplicating original test and then activating new one and paused the old test.

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

    1 reply

    PratheepArunRaj
    Community Advisor and Adobe Champion
    PratheepArunRajCommunity Advisor and Adobe Champion
    Community Advisor and Adobe Champion
    August 16, 2023

    Dear @ramya_sr ,

    Guess we already had this discussion, however, let me ask you a few questions.

    Where are you seeing this 'sample size mismatch' error/insight? Are you using some other tool to analyze this? If yes, kindly let me know.

    Besides, check out this discussion as well.

    Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

    Thank You, Pratheep Arun Raj B (Arun) | Xerago | Terryn Winter Analytics
      R
      Ramya_SRAuthor
      Level 2
      August 16, 2023

      Hi Pratheep,

       

      We used cookie drop for target setup. Only Adobe Target was assigned for reporting. This target cookie was tagged to analyse data in adobe analytics. So when calculating confidence using AB test guide , it alerted SRM.

       

       

      Although our intention might have been for the test to have a 50/50 split, if the actual split turned out to be different, like 45/55 or 40/60, could we still extract valuable insights to inform business decisions? Is it valid to disregard results just because there's uneven distribution, especially if there's sufficient traffic experiencing each variant?

       

      Your insights on this topic is highly appreciated

       

      Thanks

      Ramya

        Terms & ConditionsAccessibility statement

        Loading footer...