Where to find Standard Error in Adobe Target results?

[emmak109]

I am trying to locate the standard deviation calculated for an A/B test variants? Where can I find those in the reports?

[PratheepArunRaj]

Dear Emma,

Standard Deviation is not a direct metric in Target Activities for consumption.

Deviations are used to calculate the Confidence Level. Calculations for A/B/n activities are here.

Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

[emmak109]

Oh no! The link you shared looks so different from the current UI of Adobe Target. The only range I see in the current UI is 'range of Lift (Low, Middle, High)' - which is usually the difference in the KPI I am measuring. To be able to make correct interpretation of the test results (not just on the basis of Confidence - p values or Lift range), I need to see how much variation occurs in the normal distribution of KPI in both Control and Treatment groups. The reason I am asking this is data looks flaky to me - we have 97% confidence (0.03 pvalue) despite of extremely high range in lift (from negative: -78 to positive: 13). I need to find answer to this and that is why I wanted to look at the confidence interval range for the KPI I am using for this test. If it is not available in Adobe Target, then what workaround do you suggest?

[PratheepArunRaj · Accepted solution]

Dear @emmak109 ,

Ignore the UI, calculations are still the same. Did you go through the link shared?

The link had an option to perform the calculations offline (Attached here) and can be used. See whether it is useful to you.

Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

[emmak109]

that's the point!

I have used this sheet as a workaround, however, without knowing "Sum KPI Squared" (which in this case is Revenue) I cannot perform the standard deviations and other statistical calculation. I already used https://experienceleaguecommunities.adobe.com/t5/adobe-analytics-questions/sum-of-revenue-squared/td... to create this metric but then when I try to use this metric in A4T report I cannot breakdown the numbers for this metric by control and treatment group. How should I do this? 

[PratheepArunRaj]

Okay, got you!

Unfortunately, Calculated Metrics are not supported for 'Analytics for Target' and thus you are unable to see when you break them down by Activity.

Did you try creating a segment with Experience A/B/n separately and add the calculated metric there? Guess @Asheesh_Pandey was suggesting the same in the other thread.

Thank You, Pratheep Arun Raj B (Arun) | NextRow Digital | Terryn Winter Analytics

[emmak109]

Can you also please share the the link for that thread? I need more details on this.

[PratheepArunRaj]

You shared the link with me in the previous response.

Link: https://experienceleaguecommunities.adobe.com/t5/adobe-analytics-questions/sum-of-revenue-squared/td...

[emmak109]

Thank you for mentioning this!

I am still confused on what exact segment I need to apply and where? Are you saying after the test is completed, I should create a segment out of treatment group (variant)?

[Amelia_Waliany]

Thanks for keeping the inquiry up @emmak109, and thanks @PratheepArunRaj for your always-helpful guidance here on the thread! Wondering if any of our fellow Adobe Target Community Advisors and/ or Adobe Target Community Mentors might be able to weigh in and help clarify further?

CC our brilliant Advisors: ambikaTewari_ATCI, Andrew_Wathen_, Avinash_Gupta_,  Eric_Vidana, Gokul_Agiwal, Perrin_Ennen, Rajneesh_Gautam_, and Umamaheswari_Yakkala

and our inspiring Mentors: 

@abhilashsape, @Gokul_Agiwal, @Rajneesh_Gautam_, @PratheepArunRaj, and @Sneha-Parmar

---

Likes

Total Posts

Correct reply