Expand my Community achievements bar.

SOLVED

Adobe Alert - How do they calculated "expected" value

Avatar

Level 2

natmcd99_0-1682358724973.png

I have an alert that the daily visits were 35% above expected - how does adobe calculate the "expected" value (and thus are able to calculate the % above/below)? 

 

1 Accepted Solution

Avatar

Correct answer by
Community Advisor

One of the sessions at Adobe Connect 2023 suggested trying to use ratios (orders per visitor for example), more than raw numbers (orders) so as to receive less false alarms.

 

I believe it was in the Rockstar session if you're interested.  

Lol as I looked for the session to give the link, I realized it was Jennifer who posted above!  She's got some great ideas that she gives!  Starts around 28 minutes in:  https://business.adobe.com/summit/2023/sessions/2023-adobe-analytics-rockstars-top-tips-and-tricks-s...

View solution in original post

8 Replies

Avatar

Community Advisor and Adobe Champion

The "Expected Value" is essentially a value that is derived from your historical trends.I don't know all the math that they put into it specifically, but it will take into account the granularity of the data, probably the day of the week and time of the day... then based on the history of your data collection, and throwing out some of the outliers in the historical trends so as to not be heavily swayed by other anomalies (probably using some standard deviation rules), it determines what it believes to be a accurate representation of your data and compares the current data against it....

 

Whether or not you feel that the data is truly outside of expectations is another story entirely

 

If you are using the Anomaly Detection in Workspace, this shows you where it believes the data should be, and flags items that fall outside of it's range... personally (at least in my reports) I have never felt these to be terribly accurate or useful (but I work in News... so it's a pretty volatile market, depending on what is happening... big things happening in politics? "Feel Good" community stories that go viral? Product recall notices? There are so many factors that impact what and when people are reading, that Adobe's anomalies don't really help us out much)

 

Here is some more reading on Anomaly Detection: https://experienceleague.adobe.com/docs/analytics-platform/using/cja-workspace/virtual-analyst/anoma... which is what is also used to create the type of alert you are seeing....

Avatar

Level 2

I have some questions on the expected value shown on the line chart.

 

1.  Is the expected value fixed regardless of the date range selected?  I know moving average is changed based on the time period selected.

2.  Is there anyway to get expected value data?

3.  Is the confidence band calculated based on 95% confidence level so expected value +/- 2 standard deviation?

 

 

 

Avatar

Community Advisor and Adobe Champion

They are using some logic to determine the expected value (not that I've ever found that to match what I would consider to be "expected"... but in the news industry our traffic fluctuates greatly based on what is happening, rather than a predictable model or seasonal trend... so that likely has a lot of negative impact on these sorts of models.

 

As for the lookback for "Expected", I think that changes depending you your period. (there is more info in the link below)

 

I am not sure if the "expected" value is readily available to get as a value.. I know that in Workspace, you can see the expected value on hovering, but I don't know if there is an actual way to pull the value out for viewing on its own. You could probably create your own calculated metric to replicate it?

 

Yes, I would expect the +/- 2 standard deviation for a 95% threshold.

https://experienceleague.adobe.com/en/docs/analytics/components/alerts/intellligent-alerts 

Avatar

Level 2

Hi Jennifer,

 

Thank you very much for sharing your knowledge and understanding on my questions.  I am in ecommerce/retail. YoY trend and period over period trends tend to be consistent besides the holiday peak.  The expected value seems reasonable to me for the few periods I looked at.

 

 

 

Avatar

Community Advisor and Adobe Champion

Yes, retail tends to be more consistent. You have your standard traffic, your peaks for various sales through the year... and maybe some unexpected highs when a new "hot" product is released.

Avatar

Correct answer by
Community Advisor

One of the sessions at Adobe Connect 2023 suggested trying to use ratios (orders per visitor for example), more than raw numbers (orders) so as to receive less false alarms.

 

I believe it was in the Rockstar session if you're interested.  

Lol as I looked for the session to give the link, I realized it was Jennifer who posted above!  She's got some great ideas that she gives!  Starts around 28 minutes in:  https://business.adobe.com/summit/2023/sessions/2023-adobe-analytics-rockstars-top-tips-and-tricks-s...

Avatar

Community Advisor and Adobe Champion

Lol, yup.. that was me  

 

I focused on custom rates (where we make the math) as opposed to the Adobe provided thresholds... but play around with both! 

Avatar

Community Advisor

This year in the Rockstar session, someone told a wonderful joke that I've used several times.  I don't think it was you but perhaps Mandy George?  She was presenting on attribution IQ and it went something like:  "I was going to tell a joke about attribution...but I didn't want to take all the credit!"

Haha, I love corny jokes and think I laughed out loud when I heard that!