India Hackathon: Using MAGIX To Secure Retailers Next-Click Personalization In Milliseconds

[#1]

Authors: Pinkesh Badjatiya, Nikaash Puri, Balaji Krishnamurthy, and Per Andreasen

At the Adobe Hackathon 2019, a team of Adobe developers started working towards a solution to let retailers use Adobe Experience Platform for real-time personalization.

Despite all the data available today, even some of the largest retailers in the world are missing out on potential revenue because of slow personalization on their websites. Many retailers can only hope that website visitors who leave without making a purchase will return later and give them the opportunity to deliver a more personalized experience that might lead to a sale. There are two reasons that make real-time personalization challenging:

1. Most personalization models today are built and updated in batch mode and can, therefore, take hours to account for recent user interactions.
2. Personalization models deployed for real-time scoring often use aggregated events for each user. Aggregating these events to build a feature vector for real-time inference can be a costly operation.

Adobe Media and Data Science Research Labs teams are working on innovations to deliver a solution that enables next-click personalization. The team demonstrated this solution during the Adobe India Hackathon 2019. The solution uses a proprietary model interpretation algorithm called Model Agnostic Globally Interpretable Explanations (MAGIX) to extract rules that explain complex black-box model behavior. These rules help marketers understand patterns the model uses to make decisions and are used as streaming segments on Adobe Experience Platform to secure real-time next-click personalization.

How the solution achieves real-time personalization

The solution works in two steps. First, it uses MAGIX to extract a set of rules from a trained machine learning model. Second, it deploys these rules using the streaming segmentation service on the Adobe Experience Platform. The streaming segments are automatically updated by Adobe Experience Platform in real-time whenever a user has a new experience event. The updated segments are used to personalize the next user experience on the website. In this way, their solution uses a machine learning model to personalize the next click of a user in under 10 milliseconds.

Figure 1: Traditional inference flow (top) compared to the real-time solution (bottom)

Rather than using a machine learning model directly at inference time, which is slow, the solution uses MAGIX to deploy the model as a set of streaming segments that are updated in milliseconds.

During the Adobe India Hackathon 2019, the Adobe Media & Data Science Research team tested the feasibility of their solution for next-click personalization on Adobe Experience Platform. The same team originally invented Model Agnostic Globally Interpretable Explanations (MAGIX) about a year earlier to combat the lack of transparency in complex decisions made by non-linear functions like neural networks, Random Forests and gradient boosted trees. MAGIX provides transparency in any complex machine learning model and makes marketers aware of the foundation for the decisions drawn from it. Now, when we combine MAGIX with Adobe Experience Platform we can convert these rules into recommendations for personalization in real-time.

By translating each segmentation rule from the machine learning model into a query in Profile Query Language, each item or ad on a website can be correlated with a segment. These segments are then updated in real-time to reflect new products that the user might be interested in, as soon as the user interacts with something on the site. When this is in place, retailers need no additional infrastructure except Adobe Experience Platform to gain real-time personalization. This gives retailers interpretable rules for the inference of their machine learning model of choice. They can then use these rules for documentation work with the General Data Protection Regulation (GDPR) or segmenting manually for sensitive topics like politics.

Preempting browse abandonment

From 2010 to 2020 retailers around the globe have seen between 64–70% of their visitors leave their online stores with items left in their carts. A lot of innovation has been made to retarget those cart-abandoning visitors to get the sale through social media marketing and email marketing. But statistics show that only around 10% of website visitors make it as far as adding something to their carts in the first place.

Most online shoppers that abandon a cart do so because of price, payment issues, or bad user experience. With this solution, individual retailers can preempt the loss of trust due to bad user experiences. Improved and personalized user experiences should also lead to even more of the browsing visitors adding something to their cart and either making a purchase or being retargeted after abandoning a cart. The potential for reducing browser abandonment through better personalization is the reason why this work on a real-time next-click personalization available as a click-to-activate service on Adobe Experience Platform was voted most business impact hack at the Adobe India Hackathon 2019.

Apart from the improved speed and revenue lift, the combination of MAGIX and Adobe Experience Platform should also grant each retailer the possibility of adding human decisions to the personalization engine. This will be particularly valuable when there is not much user data to work with. The retailer can manually assign each visitor into a segment, which will then help the machine learning model reach a more precise model inference at its own pace.

The benefit of speed is precision

Deploying machine learning models for next-click personalization is difficult today because the model inference typically takes more than two minutes. The average website visitor has, therefore, left the website before the personalized experience is updated.

We needed to bring the time for user data to pass through machine learning models down to what feels like real-time to any human. In order to interpret the models and distill them into rules to speed up the personalization, we converted a trained machine personalization model into a set of rules using MAGIX.

MAGIX helps us see the mechanics of black-box machine learning by interpreting actions into rules and sorting them by importance.

Now when a user interacts with an item on a website, an Experience Event is sent to Adobe Experience Platform. This lets us see explanations for the black-box machine learning model when we distill the model into rules. For each rule, we created a Streaming Segment written in the Profile Query Language.

All segments enabled for streaming computation are updated in real-time, so when the user clicks on to the next page on the website, the updated segments make sure the website shows content that the user is more likely to interact with. This model can also be used to personalize image content in email deliveries using Adobe Campaign.

But the high speed comes at a cost. When we convert the model into rules, we lose a bit of precision. Adobe Experience Platform only allows for binary segments. Therefore users need to be either in or out of a segment, and it is difficult to rank segments as more important than others for a particular user since the segments can only be scored as “1” or “0”. That is a limitation that we will continue working on to solve.

There is also some loss of information when we create a number of rules on the inference engine. We can come pretty close to matching the accuracy and predictive power of the original machine learning model, and our solution will make it possible for the retailer to track back and detect which rule fired for the individual website visitor. A lot of industries will find it advantageous to be able to audit what made the model take a step towards further personalization.

Adobe Experience Platform offers outstanding documentation

Like every other team that took part in the Adobe India Hackathon 2019 we have our own area of expertise and had not worked in-depth with Adobe Experience Platform before. Therefore we needed accurate documentation to help us achieve as much as possible at the event. In general, we should be able to take good documentation for granted. When you are in need of help, finding answers quickly and easily is crucial to your success on any platform. Unfortunately, that is not our experience with most platforms. This is one of the many reasons why Adobe Experience Platform is a real pleasure to work with. Leading up to and during the Hackathon, we had set aside a lot of time for work on this solution, and every time we turned to the documentation, it actually reflected the way things worked. So we were able to set up unified profiles, streaming segmentation, and much more with no outside help.

When we were at the Hackathon the team behind Adobe Experience Platform was very nice and patient with all our questions. It was a big help to have them there ready to help us during the most intense part of our work. In other hackathons we have attended, we had to bootstrap everything based on our own experience and skills. Having just a few days to work solely on Adobe Experience Platform added pressure, but it also helped open our eyes to the possibilities on the platform. We had some good talks with the other attendees at the Hackathon and are now collaborating with the Adobe Experience Platform Bangalore team to share experiences about real-time scoring and push on towards working demos.

Most of the services in our next-click personalization solution are already available on Adobe Experience Platform. When our solution is produced and available it will come with our MAGIX and only take you a single click to deploy the model as rules. We don’t have a complete roadmap as of this writing. But, we plan to continue to evaluate some possibilities for a proof of concept in collaboration with other teams and colleagues.

Follow the Adobe Experience Platform Community Blog for more developer stories and resources, and check out Adobe Developers on Twitter for the latest news and developer products. Sign up here for future Adobe Experience Platform Meetups.

Resources

1. Adobe India Hackathon 2019
2. Model Agnostic Globally Interpretable Explanations (MAGIX): https://arxiv.org/pdf/1706.07160.pdf
3. Streaming Segments: https://www.adobe.io/apis/experienceplatform/home/profile-identity-segmentation/profile-identity-seg...
4. Adobe Experience Platform: https://www.adobe.com/experience-platform.html
5. Machine learning: https://www.adobe.com/dk/sensei.html
6. Neural networks: https://en.wikipedia.org/wiki/Artificial_neural_network
7. Random forests: https://docs.adobe.com/content/help/en/target/using/activities/automated-personalization/algo-random...
8. Gradient boosted trees: https://en.wikipedia.org/wiki/Gradient_boosting
9. Profile Query Language: https://docs.adobe.com/help/en/experience-platform/segmentation/pql/overview.htm
10. General Data Protection Regulation (GDPR): https://ec.europa.eu/info/law/law-topic/data-protection_en
11. Online Retail Statistics from Statista: https://www.statista.com/statistics/477804/online-shopping-cart-abandonment-rate-worldwide/
12. Abandoned Cart Statistics from Smart Insights: https://www.smartinsights.com/ecommerce/ecommerce-analytics/typical-average-add-cart-conversion-rate...
13. Experience Event: https://docs.adobe.com/content/help/en/experience-platform/xdm/home.html
14. Adobe Campaign: https://www.adobe.io/apis/experiencecloud/campaign.html
15. Real-time Unified Profile: https://www.adobe.com/experience-platform/real-time-customer-profile.html

Originally published: May 8, 2020