Changing goal metric half way through a test

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

25-06-2019

hi, i have setup an AB test based on the primary goal being activity conversions (clicks).

but - my KPI is actually bounce rate - and crucially, I want to see if the difference in bounce rate is statistically significant (which can only be done within Target if it's set as the primary goal, not in A4T).

if i change the primary goal in Target to bounce rate while the test is still live, will the bounce rate data be backdated from the start of the test or will it only happen from the point of change?

thanks

Accepted Solutions (1)

Accepted Solutions (1)

Avatar

Avatar

ryanr701

Employee

Avatar

ryanr701

Employee

ryanr701
Employee

26-06-2019

That is correct. We don't provide lift and confidence on calculated metrics - expect RPV/AOV. You have to use the raw counting metrics to see lift and confidence.

Answers (9)

Answers (9)

Avatar

Avatar

Andrey_Osadchuk

MVP

Avatar

Andrey_Osadchuk

MVP

Andrey_Osadchuk
MVP

25-06-2019

I presume it depends on how you set the Goal. If the reporting source was set to Adobe Analytics and the metric was set to a Target one, then the new metric will be available (retroactive). However, if Adobe Target was set as a source, then the new metric availability will be as of the change activation.

Avatar

Avatar

ryanr701

Employee

Avatar

ryanr701

Employee

ryanr701
Employee

25-06-2019

Hi simonb23491113​,

Regarding your comment about seeing statistical significance only on the primary goal metric. Do you already have some secondary metrics configured for the test? If so, each of the metrics can be used to calculate lift and confidence. However, you have "move" the lift/confidence columns to the desired metric. You do this by hovering over the new metric's header and clicking the arrow icon.

liftConfidenceMetricMove.png

To answer you original question: changing your primary goal metric in the middle of a test would muddy up your results. It would not be a retroactive change.

Hope that helps,

Ryan

Avatar

Avatar

ryanr701

Employee

Avatar

ryanr701

Employee

ryanr701
Employee

27-06-2019

ah...I hadn't realized you were talking about Analytics as the reporting source (A4T). You only need to set secondary metrics if you are using Target as the reporting source. With A4T all other Analytics metrics are already usable so you don't need to "set" secondary metrics - it's already done.

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

27-06-2019

hi ryanr8​ - how do i set the additional goals?

see response above "26-Jun-2019 03:54"

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

27-06-2019

thanks ryanr8!

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

26-06-2019

thanks again, done that.

selected new metric, this time conversion rate, and as you can see the usual columns for lift/confidence won't display.

is this because these are rates ?

1779911_pastedImage_0.png

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

26-06-2019

Thanks Ryan. This is massively helpful!

But how do I set secondary metrics (i can add in hidden metrics within the AT report, as below)

1779888_pastedImage_2.png

but when i setup the AT, there's only the option of the primary being set. shot below.

1779887_pastedImage_0.png

I've found out how to add other metrics (as the first screenshot) but when i add these into AT report it seems none of the "rates" let you se lift/confidence i.e. conversion rate/bounce rate/exit rate.

it seems single metric numbers can i.e. orders/bounces/exits though.

is this correct?

1779889_pastedImage_9.png

Avatar

Avatar

Andrey_Osadchuk

MVP

Avatar

Andrey_Osadchuk

MVP

Andrey_Osadchuk
MVP

26-06-2019

Simon,

Can you see a link referring to Adobe Analytics on the Reports tab? If yes, click and share what you'll see.

Avatar

Avatar

simonb23491113

Avatar

simonb23491113

simonb23491113

26-06-2019

thanks Andrey. Reporting source is set to AA. I can get the other metrics up in AA and AT - but none let me show the statistical sign (lift/confidence etc). simon