Prediction Score for Auto Target AI Algorithm

[emmak109]

Hi all, I have a question from the reporting side. When we share AT findings with the stakeholders, they want to know the accuracy of these models for every optimal experience identified for a segment. In AB activity, we have Confidence (pvalue T-Test) to understand the impact. Which accuracy score do we use in the case of Auto Target? 

[Devyendar]

@emmak109 

AT (Auto-Target) delivers personalization using ensemble machine learning models (logistic regression + decision trees + bandit learning) to predict the best experience per user based on behavioral and contextual signals (not a fixed A/B/n split), the classic "confidence" (p-value between variants of experiences doesn't apply. Adobe suggest to evaluate the outcome based on uplift vs. control with confidence rather then a stand-alone classifier accuracy.

Refer to:

Auto Target reporting: https://experienceleague.adobe.com/en/docs/target/using/activities/auto-target/reporting-and-auto-ta... 
Auto-Target Overview: https://experienceleague.adobe.com/en/docs/target/using/activities/auto-target/auto-target-to-optimi... 

[Devyendar]

@emmak109 I hope my response to your question was helpful, please mark it as "Correct Answer" so it can help other community members as well.