Advice on fairest metric for AB test (content position) | Community
Skip to main content
simonb23491113
simonb23491113
Level 2
August 2, 2019

Advice on fairest metric for AB test (content position)

  • August 2, 2019
  • 2 replies
  • 9996 views

Hiya

We're about to run a test on a key landing page to which we're we’re moving the position of an identical asset ("sale now on") lower down the page vs the control (at the top).

The objective is to understand what impact this has on performance i.e. if we should lead with a sale message or not.


What do you feel is the fairest metric to measure success?

I feel click metric won’t really tell us anything i.e.higher = more clicks.

We could use revenue metric but this will be skewed as Sale has lower price point = lower revenue.

We could use conversion but that is also quite broad.

anyone have any advice here?

Thanks

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

    2 replies

    Andrey_Osadchuk
    Andrey_Osadchuk
    Level 10
    August 2, 2019

    Hi Simon,

    Once you have an answer on what the performance is, it will be clear what metric to choose as a goal.

    Gaureshk_Kodag
    Adobe Employee
    Gaureshk_KodagAdobe Employee
    Adobe Employee
    August 5, 2019

    hi Simon,

    You could use an engagement metric to check whetver the test is impacting user to spend less time on page.

    If you want to check clicks, you can use conversion metric > Clicked an element.

    But calculating only click conversion won't be 100% useful if large amount of clicks are not getting converted into final goal. for tracking final goal after clicked an element you can use advance settings as shown in following screenshot.

    For example, If i want to measure clicks on Buy now button, I would get high conversion. But I'm not able to understand how many people actually baught the product.

    In Such case you can use advance settings as follow.

    Set primary goal as clicked an element (i.e. Buy now CTA)

    Add Secondary goal As Conversion > Viewed a page > url contains > thankyou (which is my final page after produt purchased)

    click on three vertical dots where you can see Advance settings.

    Select primary goal from selectbox for following question. (which means views on thankyou page (secondary goal) will increment only when primary goal is successful)

    Which success metrics must be reached before incrementing this metric?

    This will tell you how many people visited thankyou page after clicking on Buy now CTA.

    I hope this helps you

    Thanks,

    Gauresh Kodag

    https://www.bajajfinservmarkets.in/emi-storeAAdvanced

      MihneaD
      Adobe Employee
      MihneaDAdobe Employee
      Adobe Employee
      August 5, 2019

      That was very comprehensive!

      Mihnea Docea | Technical Support Consultant | Customer Experience | Adobe | (:: 1 (800) 497-0335

        Terms & ConditionsAccessibility statement

        Loading footer...