Is there any way to check whether AB test is having flicker effect? | Adobe Higher Education
Skip to main content
Level 2
August 11, 2023
解決済み

Is there any way to check whether AB test is having flicker effect?

  • August 11, 2023
  • 1 の返信
  • 721 ビュー

Traffic is split 50/50% between control and variation. The data analysis shows sample size mismatch. 

このトピックへの返信は締め切られました。
ベストアンサー PratheepArunRaj

Dear Ramya,

When you say flicker, are you saying that Control is loading more due to the flicker creating the size mismatch?

If yes, it should not be the case since the traffic is counted only when the offer is delivered (Either it is control or variation).

The unequal split is common during the initial days and settles over a week. Just continue with the test and see whether the numbers are normalizing in a week or so. If not, reach us back.

Thank You, Pratheep Arun Raj B (Arun) | NextRow DigitalTerryn Winter Analytics

1 の返信

PratheepArunRaj
Community Advisor and Adobe Champion
Community Advisor and Adobe Champion
August 14, 2023

Dear Ramya,

When you say flicker, are you saying that Control is loading more due to the flicker creating the size mismatch?

If yes, it should not be the case since the traffic is counted only when the offer is delivered (Either it is control or variation).

The unequal split is common during the initial days and settles over a week. Just continue with the test and see whether the numbers are normalizing in a week or so. If not, reach us back.

Thank You, Pratheep Arun Raj B (Arun) | NextRow DigitalTerryn Winter Analytics

Thank You, Pratheep Arun Raj B (Arun) | Xerago | Terryn Winter Analytics