Is there any way to check whether AB test is having flicker effect?

1 reply
737 views
Translate All

Traffic is split 50/50% between control and variation. The data analysis shows sample size mismatch.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

PratheepArunRajAccepted solution

Adobe Champion, Community Advisor, UG Leader

Dear Ramya,

When you say flicker, are you saying that Control is loading more due to the flicker creating the size mismatch?

If yes, it should not be the case since the traffic is counted only when the offer is delivered (Either it is control or variation).

The unequal split is common during the initial days and settles over a week. Just continue with the test and see whether the numbers are normalizing in a week or so. If not, reach us back.

Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

Thank You, Pratheep Arun Raj B (Arun) | Xerago | Terryn Winter Analytics

PratheepArunRaj

Accepted solution

Adobe Champion, Community Advisor, UG Leader

Dear Ramya,

When you say flicker, are you saying that Control is loading more due to the flicker creating the size mismatch?

If yes, it should not be the case since the traffic is counted only when the offer is delivered (Either it is control or variation).

The unequal split is common during the initial days and settles over a week. Just continue with the test and see whether the numbers are normalizing in a week or so. If not, reach us back.

Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

Translate

Thank You, Pratheep Arun Raj B (Arun) | Xerago | Terryn Winter Analytics

Sign up

Login with SSO

Login to the community

Login with SSO