How Adobe Experience Platform Predictive Audiences Improves Personalized Experiences

[#1]

Authors: Matt Kipilevich and Margarita Savova

Predictive Audiences was born out of a Garage Week project by the authors of this blog. Once a year, for one week, Adobe grants engineers complete freedom to work on an idea that interests them. Some of the most innovative ideas across Adobe’s Digital Experience business unit were born out of Garage Week projects, such as ours.

When we started, we knew that our customers were spending a lot of time manually creating segments to deliver relevant experiences for the personas they had in mind. We also acknowledged that in many cases, they had a very small window to engage with their users and offer them a meaningful personalized experience, since many anonymous third-party cookies have a lifespan of just one session. We aimed to help our customers by leveraging Machine Learning and creating a system that would allow them to offer effective on-click personalization the instant they saw a user on their property, minimizing guesswork and reducing manual labor.

We are happy to say that we have achieved our vision and as of May 2020 Predictive Audiences is available for all Adobe Experience Platform Audience Manager customers. Over one hundred customers are using Predictive Audiences today to take the user’s personalization experience to the next level.

Predictive Audiences enables customers to:

• Classify users into distinct audiences in order to personalize across channels and devices on-click in real-time.
• Improve intelligence and personalization for authenticated and anonymous audiences.
• Guarantee that a user is placed into one and only one persona and therefore Predictive Audience segments have no overlapping users.
• Improve CPA.
• Increase their users’ CLTV (Customer Lifetime Value).
• Automate the process of building quantitative high-quality audiences, eliminating manual and heuristic segmentation.

Predictive Audience segments fully support real-time Profile Merge that allows marketers to control which profiles are being used for classification. The feature is also fully integrated with Audience Manager’s Data Export Controls to ensure that the Predictive Segments are never sent to destinations that would violate data privacy or data use agreements.

The Workflow

Imagine that you are the Digital Marketing Manager at adobe.com and your goal is to offer on-site personalization in order to improve user engagement. You have already built audiences from known users, who have purchased different Creative Cloud products related to Photography, Graphic Design, Video, etc. Your objective is to determine which of these personas your website visitor most closely aligns to. Let’s examine how you can achieve this marketing goal with Predictive Audiences in just a few simple steps.

The first step is to select a set of base personas. Typically, these are highly valuable customer audiences, such as conversion traits, or users that have declared a specific interest.

Figure 1: Selecting Base Personas

The second step is to select a target audience, such as everyone who comes to your site. Users from the target audience, who are not already part of a base persona, will be classified in real-time into one of the base personas. In your case, this will be the adobe.com site visitor trait.

Figure 2: Selecting Target Audience

The third step is to provide additional model details, such as Name, Description, and a Profile Merge Rule. The latter defines which of the user’s device(s), and/or authenticated profiles should be used when classifying them into a Predictive Segment. It also governs which trait types can be used during the model training step.

Figure 3: Finalizing Model Details and Saving It

As soon as you hit the <Save> button, the system automatically provisions Predictive Audience Segments, one for each of the base personas selected. The model is then immediately sent for training. Within about an hour, it is pushed out to the Data Collection Servers. Once loaded into memory, the model begins placing visitors into the newly created Predictive Segments in real-time.

Reporting metrics such as a classification breakdown between the Predictive Segments, as well as real-time population trend graphs, help you understand the reach of your model.

Figure 4: Viewing Model Results

Technology Under the Hood

Model Interpretability

Predictive Audiences gives the marketers full transparency into their models by letting them view the weights that were assigned for each trait. We picked Logistic Regression because it performs consistently well across a wide variety of customers and data patterns, and also lends itself to easy interpretation. When using Predictive Audiences, marketers are putting their trust in the models. They need to see that what the models have learned is meaningful and free of biases.

An important note about the weights that Logistic Regression assigns is that they are expressed in terms of log-odds. A positive weight means it would increase the likelihood of a user belonging to that persona. A negative weight means it would decrease the likelihood of a user belonging to that persona. For a more detailed explanation, please see the following.

Figure 5: Viewing Influential Traits

Feature Selection

Predictive Audiences looks through each trait a marketer has defined in the system and automatically selects the best ones for a given model. The model defines “best” as having the highest F1 score, which is the harmonic average of precision and recall. Intuitively, the model picks traits that are both highly predictive for a given persona (have high precision) and have a large number of users who qualify for that persona (have high recall).

To compute the true positives for a given trait with respect to each persona, which is needed to compute precision and recall, we have to compute the intersection of users who qualify for both the persona segment and the predictive trait. To compute this intersection efficiently, we use MinHash signatures right before the model is trained.

Model Tuning

Predictive Audiences is used by many customers with different data patterns. To ensure that each model is optimally tuned, we vary L1 regularization penalty and select one that yields the highest test accuracy. L1 regularization has a nice property of zeroing out traits with small weights, thus helping us reduce model complexity and helping ensure that the model generalizes better.

Figure 6: Simplified Architectural Diagram

Model Retraining and Evaluation

To ensure that the latest data is always analyzed, models are retrained daily. Their real-world performance is also measured and tracked on a daily basis via MinHash signatures. The signatures help us determine over time whether we have made the correct predictions, i.e. if people are moving from a predictive audience into its corresponding persona.

What’s Next

Even though Predictive Audiences has already proven to be extremely valuable, based on our customers’ feedback and the skyrocketing adoption we have seen since GA, we are committed to keep enhancing it. We are continuously improving models’ performance and making UI enhancements. We plan on adding backend segmentation support so that marketers can classify historic audiences without having to wait for visitors to appear on their sites, and use the Predictive Audience segments for off-site retargeting. We are also looking for ways to provide guidance to our customers during the model building phase and offer them early feedback about the quality of the model inputs.

Follow the Adobe Experience Platform Community Blog for more developer stories and resources, and check out Adobe Developers on Twitter for the latest news and developer products. Sign up here for future Adobe Experience Platform Meetups.

References

1. On Densification for Minwise Hashing: http://auai.org/uai2019/proceedings/papers/302.pdf
2. One Permutation Hashing: https://www.stat.rutgers.edu/home/pingli/papers/DensifyOnePermutationHashingNIPS2012ICML2014UAI2014....
3. Optimal densification for fast and accurate minwise hashing: https://dl.acm.org/doi/10.5555/3305890.3306007
4. Scikit-learn: https://scikit-learn.org/stable
5. Apache Spark: https://spark.apache.org
6. AWS Batch: https://aws.amazon.com/batch
7. Predictive Audiences Documentation: https://docs.adobe.com/content/help/en/audience-manager/user-guide/features/algorithmic-models/predi...
8. Predictive Audiences FAQ: https://docs.adobe.com/content/help/en/audience-manager/user-guide/faqs/faq-predictive-audiences.htm...
9. Tutorial: Understanding Predictive Audiences: https://docs.adobe.com/content/help/en/audience-manager-learn/tutorials/build-and-manage-audiences/a...
10. Tutorial: Configure and Report on Predictive Audiences in Audience Manager: https://docs.adobe.com/content/help/en/audience-manager-learn/tutorials/build-and-manage-audiences/a...
11. Profile Merge Rule: https://docs.adobe.com/content/help/en/audience-manager/user-guide/features/profile-merge-rules/merg...
12. Data Export Controls: https://docs.adobe.com/content/help/en/audience-manager/user-guide/features/data-export-controls.htm...

Originally published: Aug 19, 2020