Traffic a A/B split test not splitting as set

[maureenb1015]

I've set up an A/B split test routing traffic 50/50 to a redirect page. But after activating the test and letting it run for a few days reporting shows the split of 93/7. Why is it not splitting as set?

[pranavkaushik]

Hi Maureen,

I suspect that you might have activated,deactivated and reactivated test before letting it remain live for some time and also the case might be that you have changed the split in between.

These were usually the reasons behind this skewedness.

Could you please confirm.

[maureenb1015]

pranavkaushik​ - thanks for responding. For this experiment I did not activate, deactivate and reactivate. Only activated it once and its been running since. Any other ideas?

[pranavkaushik]

Hi ,

Also what about the change in split distribution in between i.e. after the test has been made live did you change the traffic split??

[maureenb1015]

Launched at 50/50 - didn't change the split after launch

[LJ_Jones]

A couple of thoughts.

1. How much traffic are you talking about here?  Can you give us some actual numbers?  Sometimes with small volumes of traffic the split takes a bit of time to even out, even though it shouldn't.
2. Do you have any other audience segments set up?  Sometimes audience segments can cause strange splits.

[maureenb1015]

@LJ Jones - over 12k uniques have been through the experiment so that's not the issues. And no other audience segments are set up so no one should be suppressed from either cell.

I am getting A4T will only work with redirect offers with at.js v.0.9.6 or newer and visitorAPI.js v2.1.0 or newer. Learn more. warning but I would assume if this was an issue no one would have gotten into the test cell.

[LJ_Jones]

We probably should have started with reporting.  Where are you looking at reporting?  In A4T(Analytics) or in Target?  The error you are getting is telling you that its a reporting error and not a splitting traffic error.  So people can still get into the experience, but it will report wrong in A4T if you are on an older version of at.js.  Do you know what version you are on?  And have you checked to see what Target is reporting?

[maureenb1015]

@LJ Jones - Looking at reporting in Target but using "Orders" which is an Analytics metric to measure success. However, the visitor traffic split is coming from Target and is 93/7 so not sure what could be wrong. Anyway, I've stopped the test and am going to build a new one to start over and see if that will work. Just frustrating as its a simple A/B split test. Thanks for your input.