Changing goal metric half way through a test

[#1]

hi, i have setup an AB test based on the primary goal being activity conversions (clicks).

but - my KPI is actually bounce rate - and crucially, I want to see if the difference in bounce rate is statistically significant (which can only be done within Target if it's set as the primary goal, not in A4T).

if i change the primary goal in Target to bounce rate while the test is still live, will the bounce rate data be backdated from the start of the test or will it only happen from the point of change?

thanks

[#2 · Accepted solution]

That is correct. We don't provide lift and confidence on calculated metrics - expect RPV/AOV. You have to use the raw counting metrics to see lift and confidence.

[#3]

I presume it depends on how you set the Goal. If the reporting source was set to Adobe Analytics and the metric was set to a Target one, then the new metric will be available (retroactive). However, if Adobe Target was set as a source, then the new metric availability will be as of the change activation.

[#4]

Hi simonb23491113​,

Regarding your comment about seeing statistical significance only on the primary goal metric. Do you already have some secondary metrics configured for the test? If so, each of the metrics can be used to calculate lift and confidence. However, you have "move" the lift/confidence columns to the desired metric. You do this by hovering over the new metric's header and clicking the arrow icon.

To answer you original question: changing your primary goal metric in the middle of a test would muddy up your results. It would not be a retroactive change.

Hope that helps,

Ryan

[#5]

ah...I hadn't realized you were talking about Analytics as the reporting source (A4T). You only need to set secondary metrics if you are using Target as the reporting source. With A4T all other Analytics metrics are already usable so you don't need to "set" secondary metrics - it's already done.

[#6]

hi ryanr8​ - how do i set the additional goals?

see response above "26-Jun-2019 03:54"

[#7]

thanks ryanr8!

[#8]

thanks again, done that.

selected new metric, this time conversion rate, and as you can see the usual columns for lift/confidence won't display.

is this because these are rates ?

[#9]

Thanks Ryan. This is massively helpful!

But how do I set secondary metrics (i can add in hidden metrics within the AT report, as below)

but when i setup the AT, there's only the option of the primary being set. shot below.

I've found out how to add other metrics (as the first screenshot) but when i add these into AT report it seems none of the "rates" let you se lift/confidence i.e. conversion rate/bounce rate/exit rate.

it seems single metric numbers can i.e. orders/bounces/exits though.

is this correct?

[#10]

Simon,

Can you see a link referring to Adobe Analytics on the Reports tab? If yes, click and share what you'll see.

[#11]

thanks Andrey. Reporting source is set to AA. I can get the other metrics up in AA and AT - but none let me show the statistical sign (lift/confidence etc). simon