Changing goal metric half way through a test | Community
Skip to main content
simonb23491113
simonb23491113
Level 2
June 25, 2019
Solved

Changing goal metric half way through a test

  • June 25, 2019
  • 10 replies
  • 11323 views

hi, i have setup an AB test based on the primary goal being activity conversions (clicks).

but - my KPI is actually bounce rate - and crucially, I want to see if the difference in bounce rate is statistically significant (which can only be done within Target if it's set as the primary goal, not in A4T).

if i change the primary goal in Target to bounce rate while the test is still live, will the bounce rate data be backdated from the start of the test or will it only happen from the point of change?

thanks

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by ryanr7

    That is correct. We don't provide lift and confidence on calculated metrics - expect RPV/AOV. You have to use the raw counting metrics to see lift and confidence.

    10 replies

    Andrey_Osadchuk
    Andrey_Osadchuk
    Level 10
    June 25, 2019

    I presume it depends on how you set the Goal. If the reporting source was set to Adobe Analytics and the metric was set to a Target one, then the new metric will be available (retroactive). However, if Adobe Target was set as a source, then the new metric availability will be as of the change activation.

      R
      Adobe Employee
      ryanr7Adobe Employee
      Adobe Employee
      June 25, 2019

      Hi simonb23491113​,

      Regarding your comment about seeing statistical significance only on the primary goal metric. Do you already have some secondary metrics configured for the test? If so, each of the metrics can be used to calculate lift and confidence. However, you have "move" the lift/confidence columns to the desired metric. You do this by hovering over the new metric's header and clicking the arrow icon.

      To answer you original question: changing your primary goal metric in the middle of a test would muddy up your results. It would not be a retroactive change.

      Hope that helps,

      Ryan

        simonb23491113
        simonb23491113Author
        Level 2
        June 26, 2019

        thanks Andrey. Reporting source is set to AA. I can get the other metrics up in AA and AT - but none let me show the statistical sign (lift/confidence etc). simon

        Andrey_Osadchuk
        Andrey_Osadchuk
        Level 10
        June 26, 2019

        Simon,

        Can you see a link referring to Adobe Analytics on the Reports tab? If yes, click and share what you'll see.

        simonb23491113
        simonb23491113Author
        Level 2
        June 26, 2019

        Thanks Ryan. This is massively helpful!

        But how do I set secondary metrics (i can add in hidden metrics within the AT report, as below)

        but when i setup the AT, there's only the option of the primary being set. shot below.

        I've found out how to add other metrics (as the first screenshot) but when i add these into AT report it seems none of the "rates" let you se lift/confidence i.e. conversion rate/bounce rate/exit rate.

        it seems single metric numbers can i.e. orders/bounces/exits though.

        is this correct?

        simonb23491113
        simonb23491113Author
        Level 2
        June 26, 2019

        thanks again, done that.

        selected new metric, this time conversion rate, and as you can see the usual columns for lift/confidence won't display.

        is this because these are rates ?

        R
        Adobe Employee
        ryanr7Adobe EmployeeAccepted solution
        Adobe Employee
        June 26, 2019

        That is correct. We don't provide lift and confidence on calculated metrics - expect RPV/AOV. You have to use the raw counting metrics to see lift and confidence.

          simonb23491113
          simonb23491113Author
          Level 2
          June 27, 2019

          thanks ryanr8!

          simonb23491113
          simonb23491113Author
          Level 2
          June 27, 2019

          hi ryanr8​ - how do i set the additional goals?

          see response above "26-Jun-2019 03:54"

          R
          Adobe Employee
          ryanr7Adobe Employee
          Adobe Employee
          June 27, 2019

          ah...I hadn't realized you were talking about Analytics as the reporting source (A4T). You only need to set secondary metrics if you are using Target as the reporting source. With A4T all other Analytics metrics are already usable so you don't need to "set" secondary metrics - it's already done.

          Terms & ConditionsAccessibility statement

          Loading footer...