Advice on fairest metric for AB test (content position)

Avatar

Avatar
Validate 1
Level 2
simonb23491113
Level 2

Likes

13 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 10
Boost 1
View profile

Avatar
Validate 1
Level 2
simonb23491113
Level 2

Likes

13 likes

Total Posts

13 posts

Correct reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 10
Boost 1
View profile
simonb23491113
Level 2

02-08-2019

Hiya

We're about to run a test on a key landing page to which we're we’re moving the position of an identical asset ("sale now on") lower down the page vs the control (at the top).

The objective is to understand what impact this has on performance i.e. if we should lead with a sale message or not.


What do you feel is the fairest metric to measure success?

I feel click metric won’t really tell us anything i.e.higher = more clicks.

We could use revenue metric but this will be skewed as Sale has lower price point = lower revenue.

We could use conversion but that is also quite broad.

anyone have any advice here?

Thanks

Replies

Avatar

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

537 likes

Total Posts

968 posts

Correct reply

263 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

537 likes

Total Posts

968 posts

Correct reply

263 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile
Andrey_Osadchuk
MVP

02-08-2019

Hi Simon,

Once you have an answer on what the performance is, it will be clear what metric to choose as a goal.

Avatar

Avatar
Springboard
Level 8
gaureshk3014423
Level 8

Likes

198 likes

Total Posts

168 posts

Correct reply

33 solutions
Top badges earned
Springboard
Validate 1
Contributor
Coach
Shape 10
View profile

Avatar
Springboard
Level 8
gaureshk3014423
Level 8

Likes

198 likes

Total Posts

168 posts

Correct reply

33 solutions
Top badges earned
Springboard
Validate 1
Contributor
Coach
Shape 10
View profile
gaureshk3014423
Level 8

05-08-2019

hi Simon,

You could use an engagement metric to check whetver the test is impacting user to spend less time on page.

If you want to check clicks, you can use conversion metric > Clicked an element.

But calculating only click conversion won't be 100% useful if large amount of clicks are not getting converted into final goal. for tracking final goal after clicked an element you can use advance settings as shown in following screenshot.

For example, If i want to measure clicks on Buy now button, I would get high conversion. But I'm not able to understand how many people actually baught the product.

In Such case you can use advance settings as follow.

Set primary goal as clicked an element (i.e. Buy now CTA)

1807197_pastedImage_10.png

Add Secondary goal As Conversion > Viewed a page > url contains > thankyou (which is my final page after produt purchased)

click on three vertical dots where you can see Advance settings.

Select primary goal from selectbox for following question. (which means views on thankyou page (secondary goal) will increment only when primary goal is successful)

Which success metrics must be reached before incrementing this metric?

This will tell you how many people visited thankyou page after clicking on Buy now CTA.

1807205_pastedImage_14.png

I hope this helps you

Thanks,

Gauresh Kodag

https://www.bajajfinservmarkets.in/emi-storeAAdvanced

Avatar

Avatar
Coach
Employee
mikewebguy
Employee

Likes

635 likes

Total Posts

402 posts

Correct reply

85 solutions
Top badges earned
Coach
Contributor
Shape 25
Shape 1
Shape 10
View profile

Avatar
Coach
Employee
mikewebguy
Employee

Likes

635 likes

Total Posts

402 posts

Correct reply

85 solutions
Top badges earned
Coach
Contributor
Shape 25
Shape 1
Shape 10
View profile
mikewebguy
Employee

05-08-2019

That was very comprehensive!

Mihnea Docea | Technical Support Consultant | Customer Experience | Adobe | (:: 1 (800) 497-0335