Adobe Target AB test not splitting traffic as set | Community
Skip to main content
A
angharadm287005
Level 2
August 15, 2022
Solved

Adobe Target AB test not splitting traffic as set

  • August 15, 2022
  • 2 replies
  • 4283 views

Hi all,

 

We've had an AB test with 1 variant running at 50:50 traffic split for the last 4 days. However we're consistently seeing more of a 52% 48% split. We aren't using auto allocate. Is there any reason for the test not being closer to 50:50?

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by Gaureshk_Kodag

    @angharadm287005 As far as I have worked on target, I have always seen a slight difference in the split at least for a few initial days which is like 48-52, 49-51. I think you should give it some time and see the change in traffic split.

    After a week or so the difference would decrease as more traffic comes in. That is normal behavior I would say.

    2 replies

    Gaureshk_Kodag
    Adobe Employee
    Gaureshk_KodagAdobe Employee
    Adobe Employee
    August 15, 2022

    In case of auto allocation, 20% of traffic is reseverd to serve random experience and 80% is served to winner experience. Now, when will traget decide who's the winner? - The algorithm starts working after all experiences in the activity have a minimum of 1,000 visitors and 50 conversions.

    In your case if you're seeing bit more traffic to experience A could be because more people are converting in exp A. Also target never split traffic in exact number that is provided in targeting settings thr is always slight difference, as it continuesly splitting traffic as it comes in.

     

    Hope this helps.

      A
      angharadm287005Author
      Level 2
      August 15, 2022

      Thank you. 

       

      We aren't using auto allocate, we are using a manual split. And the split has been consistently uneven since the test has been live. 

       

      How much slight difference would be considered within a normal range?

        Gaureshk_Kodag
        Adobe Employee
        Gaureshk_KodagAdobe EmployeeAccepted solution
        Adobe Employee
        August 16, 2022

        @angharadm287005 As far as I have worked on target, I have always seen a slight difference in the split at least for a few initial days which is like 48-52, 49-51. I think you should give it some time and see the change in traffic split.

        After a week or so the difference would decrease as more traffic comes in. That is normal behavior I would say.

          A
          Adobe Employee
          akbansalAdobe Employee
          Adobe Employee
          August 15, 2022

          There could be multiple factors affecting the traffic split among your control and challenger variants.

           

          Firstly, total traffic in the test. If the test is in the initial stage after launch and the numbers are small, you might see an unequal split. However, as the traffic increases the split should be as you have set manually in Target. Example, for a low traffic page (100 Visitors per page) you might not see an equal split initially. The overall period for the split to converge depends on traffic, and not so much on days for which the experiment has been live.

           

          Secondly, Target and Analytics have different engines to count visitors. Yes, if you are using Analytics for Target (aka A4T) numbers should be equal and there will be an equal split. If you are using a custom code(sending Control or Challenger information into a variable) to send traffic to Adobe Analytics, you might need to check if the code runs every time. As Target and Analytics have different engines to count visitors, it is recommended to verify Traffic split of A4T activities through Target activity performance APIs. APIs can be referred here >  Target API v1.0

           

          Thirdly, Visitors in an A/B test will stay in their initially-assigned experiences. The traffic change impacts new entrants only. Are you sure you don't have large number of repeat visitors?

            Terms & ConditionsAccessibility statement

            Loading footer...