Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
BedrockMission!

Learn More

View all

Sign in to view all badges

Traffic a A/B split test not splitting as set

Avatar

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile
maureenb1015
Level 2

16-08-2018

I've set up an A/B split test routing traffic 50/50 to a redirect page. But after activating the test and letting it run for a few days reporting shows the split of 93/7. Why is it not splitting as set?

Replies

Avatar

Avatar
Boost 5
Level 4
pranavkaushik
Level 4

Likes

39 likes

Total Posts

37 posts

Correct Reply

13 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile

Avatar
Boost 5
Level 4
pranavkaushik
Level 4

Likes

39 likes

Total Posts

37 posts

Correct Reply

13 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile
pranavkaushik
Level 4

16-08-2018

Hi Maureen,

I suspect that you might have activated,deactivated and reactivated test before letting it remain live for some time and also the case might be that you have changed the split in between.

These were usually the reasons behind this skewedness.

Could you please confirm.

Avatar

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile
maureenb1015
Level 2

17-08-2018

pranavkaushik​ - thanks for responding. For this experiment I did not activate, deactivate and reactivate. Only activated it once and its been running since. Any other ideas?

Avatar

Avatar
Boost 5
Level 4
pranavkaushik
Level 4

Likes

39 likes

Total Posts

37 posts

Correct Reply

13 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile

Avatar
Boost 5
Level 4
pranavkaushik
Level 4

Likes

39 likes

Total Posts

37 posts

Correct Reply

13 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile
pranavkaushik
Level 4

17-08-2018

Hi ,

Also what about the change in split distribution in between i.e. after the test has been made live did you change the traffic split??

Avatar

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

27 likes

Total Posts

37 posts

Correct Reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

27 likes

Total Posts

37 posts

Correct Reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile
LJ_Jones
Level 3

17-08-2018

A couple of thoughts.

  1. How much traffic are you talking about here?  Can you give us some actual numbers?  Sometimes with small volumes of traffic the split takes a bit of time to even out, even though it shouldn't.
  2. Do you have any other audience segments set up?  Sometimes audience segments can cause strange splits.

Avatar

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile
maureenb1015
Level 2

17-08-2018

Launched at 50/50 - didn't change the split after launch

Avatar

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile
maureenb1015
Level 2

17-08-2018

@LJ Jones - over 12k uniques have been through the experiment so that's not the issues. And no other audience segments are set up so no one should be suppressed from either cell.

I am getting A4T will only work with redirect offers with at.js v.0.9.6 or newer and visitorAPI.js v2.1.0 or newer. Learn more. warning but I would assume if this was an issue no one would have gotten into the test cell.

Avatar

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

27 likes

Total Posts

37 posts

Correct Reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile

Avatar
Boost 5
Level 3
LJ_Jones
Level 3

Likes

27 likes

Total Posts

37 posts

Correct Reply

8 solutions
Top badges earned
Boost 5
Boost 3
Boost 25
Boost 10
Boost 1
View profile
LJ_Jones
Level 3

17-08-2018

We probably should have started with reporting.  Where are you looking at reporting?  In A4T(Analytics) or in Target?  The error you are getting is telling you that its a reporting error and not a splitting traffic error.  So people can still get into the experience, but it will report wrong in A4T if you are on an older version of at.js.  Do you know what version you are on?  And have you checked to see what Target is reporting?

Avatar

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile

Avatar
Validate 1
Level 2
maureenb1015
Level 2

Likes

5 likes

Total Posts

7 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
View profile
maureenb1015
Level 2

20-08-2018

@LJ Jones - Looking at reporting in Target but using "Orders" which is an Analytics metric to measure success. However, the visitor traffic split is coming from Target and is 93/7 so not sure what could be wrong. Anyway, I've stopped the test and am going to build a new one to start over and see if that will work. Just frustrating as its a simple A/B split test. Thanks for your input.