How to efficiently/easily calculate statistical significance/confidence in adobe analytics workspace

[Venky235]

Our team does A/B testing rigorously. I am from reporting side and for the initial reporting for every A/B test, we have a adobe dashboard explaining the results with KPIs like impressions, clicks, CTR, lift and few down stream metrics for each version. Typically, we have 2-4 versions for very test - one control and 3 variants. I want to add statistical significance in each dashboard. I have calculated it using the below logic.

Imagine a test with only 2 recipes - 1 control and 1 variant

CTR 1 = clicks1/impressions1

CTR2 = clicks 2/impressions 2

Pooled CTR = (clicks 1+clicks 2)/(impressions 1+impressions 2)

Standard Error (SE) = Square root(Pooled CTR*(1-Pooled CTR)*((1/Imp1)+(1/Imp2)))

Z Score = abs(CTR1-CTR2)/SE

Standard significance = cdf-z function (Z score)

I create every calculated metric separately and finally calculate the standard significance. But this is not scalable. Because I have to create 7 calculated metrics for each test. So if a A/B test has 4 variants, I would be needing statistical significance to be calculated 3 times against the control (Which makes it 7*3=21 calculated metrics).

Is there any easy way to do this? Any efficient way of achieving this would be of great help. Thanks!

[MandyGeorge]

Unfortunately, there isn't an easy way to scale. When you're building out metrics/segments, you have to identify the specific component and component value that you want to use. For stuff like this that I need to scale to a lot of different things, I do it outside of workspace using the data feeds. SQL is a bit more flexible, and I can just say use the value of a component, and it will use the same value all the way through, and do it for each value. 

I do wish that something like this was possible in workspace. If you don't use data feeds, unfortunately you will have to build this out for each scenario, so 21 metrics. It can be a bit of a pain to do it, but once you have the first set built, you can just do an edit and save as so you aren't starting complete from scratch for each one.