Hacking Adobe Experience Platform to Create More Valuable Applications Than We Had Dared to Hope

[#1]

Authors: Manoj Thakur, Abhishek Shukla, Atul Shrivastava, Som Satapathy, Anup, Balaji K, Per Andreasen

This post describes the experiences and outcomes of an intensive hackathon put on by Adobe Experience Platform India team. Adobe engineers came together to develop five innovative applications that secure better ROI, improve UX and strengthen machine learning.

In December 2019, several developers from Adobe Experience Platform’s engineering team in India participated in an intensive internal hackathon. In just one workweek, five new and innovative platform solutions came to life.

As developers, we can easily stick to proven routines, especially in times with high workloads. We knew that Adobe Experience Platform needed to be top of mind in our daily work lives as it can take customer experience to the next level and solve problems at scale in real-time. That is why we decided to organize a hackathon for Adobe engineers in India. But we had not dared hope for as much innovation as we saw in this short time.

Five teams of passionate Adobe engineers and product managers got together to focus on their ideas for new applications on Adobe Experience Platform. Admittedly, you can’t build a new application for Adobe Experience Platform during a traditional hackathon. The standard time restriction of a full workweek at the most is too short for a small team to go through the necessary brainstorming, coding, and implementation on this complex platform. But we didn’t want to settle for another research paper or our own exciting claims about what Adobe Experience Platform can do in theory.

So how could we prove that Adobe Experience Platform enhances your capability to build an experience application at high scale? We decided to let our colleagues find some challenges that the market had with cloud solutions. And then we attempted to solve them by building new services on Adobe Experience Platform.

We wanted to make fast progress and prove the true power of Adobe Experience Platform with exciting new solutions that could quickly go to market. So the teams needed to present working code for their new concepts in order to participate in the hackathon. The outcomes were compelling innovations, which we will go over below and dive deeper into in later posts.

We invited 200 engineers from Adobe Experience Platform’s Team in India to come up with something they would like to build for the platform. This is what came out of the process.

Connecting Adobe engineers globally

About a month and a half before the actual hackathon, all the participating engineers began work on their own ideas in small teams. This made it possible to evaluate the feasibility of each technology and design through two sets of trainings. The first training was held locally and introduced each team to the full capabilities of Adobe Experience Platform. This helped the teams get clear on the feasibility of their ideas. Five teams were then selected to take part in a hands-on workshop on the first full day of the hackathon.

This two-step warm-up was crucial in order to have solutions ready for implementation in Adobe Experience Platform at the hackathon. Team leaders identified between three and four people for their team. Each team was assigned a buddy — a developer with good awareness of the possibilities in Adobe Experience Platform — and a mentor from the senior engineering staff to help with the technical solutions. These helpers, located in Adobe’s U.S. and India offices, shared their knowledge and experience with their respective teams in order to save time by pointing to proper tools and solutions on the platform ahead of the event.

Furthering the understanding of Adobe Experience Platform

The Adobe engineering team in India wanted to help more engineers experience the power of Adobe Experience Platform. Through this event, all attendees gained a thorough understanding of how to build new applications on the platform. It is now clear to our organization that we all need to have it top of mind when we help our customers imagine new solutions to their challenges.

The assistance of engineers experienced with Adobe Experience Platform gave the participants the needed insight and experience to start developing solutions for the platform on their own. And we are confident that some of the solutions developed as a result of our hackathon will become valuable product offerings for our customers in the future.

Five new ideas can become applications

Without the aforementioned strategy, we would not have ended up with any working solutions on a platform as complex as Adobe Experience Platform. Even with much of the work done before the event and the help provided to each team by engineers experienced with the platform, we did not expect all teams to achieve proof of concept before the end of the event. Our applicants came up with 11 ideas for possible applications, and from that, we selected the five that exhibited the best chances for success. Because of these preparations, our teams had working code ready for the hackathon and could work on implementation. These elements proved to form a valuable approach to a hackathon on Adobe Experience Platform.

At the end of the hackathon, each team presented their solutions and learnings. In true hackathon style, we had put together a jury to evaluate the projects. The jury was composed of five very passionate Adobe employees from research, marketing, engineering, customer delivery, and product management. Presentations were scored with points of 1–10 on possible business impact, level of innovation and level of market readiness. The jury members were clearly eager to see the applications brought to market. So we now have a possible path forward for each of the following five hackathon projects.

1. Ad currency optimization as an intelligent service on Adobe Experience Platform

Most media and entertainment publishers like Fox and NBC sell their digital TV ad inventory against specific demographics. Advertisers pay for ad impressions served on these specific demographics when these are later reported by measurement vendors like Nielsen or Comscore. This model of ad sales causes publishers to lose between 40 and 45% of their ad revenue due to the lack of demographic data on their anonymous viewers.

Adobe Primetime’s team has come up with a patented model to predict the age and gender of a user based on viewing behavior. At the hackathon, detailed viewership data about each user was provided by Adobe Analytics and passed through the model. The demographic predictive model was deployed on Data Science Workspace in Adobe Experience Platform to compute the age and gender probabilities of the individual viewer. This was then output to validation with the measurement vendors, segmentation and activation on the publisher’s ad server via real-time segment destination.

Implementing this solution as an intelligent service on the platform could make the predictions available faster to the publisher for the execution of ad campaigns with a segment transfer latency dropping from four days to near real-time. This can also reduce the onboarding time required to activate the service for a publisher by an eighth. To each publisher, this could mean between 30 and 40% revenue lift and an added $800 million to the group’s total addressable market.

2. “ExCelerate” proficiency-based user classification

When you log into Adobe Experience Platform, you are greeted with the same learning content and tutorial recommendations regardless of your level of experience and proficiency with solutions on the platform. Novice users need basic guidance and help, while experienced users can grow tired of seeing the same basic tutorials when they have completed them and really need new input to add to their skills on the platform.

That was the motivation for “ExCelerate” — a team of Adobe developers focused on optimizing the user experience — to create a proficiency-based user classification that personalizes the learning content surfaced up on the Adobe Experience Cloud home page. The solution derives user behavioral data from Adobe Analytics, streams that data into Adobe Experience Platform and leverages the process automation system to interpret user proficiency. The outcome is a better, truly personalized and effective learning experience for every user.

3. Real-time personalization at scale for next click personalization

*Voted most business impact hack*

Most personalization is not done in real-time today and certainly not through machine learning. Even the websites of global top retailers are slow to show you the products you have shown a clear interest in. The Media and Data Science Research team solved this by converting a trained machine learning personalization model into a set of rules using the model interpretation algorithm MAGIX. They deployed these rules as Streaming Segments written in the Profile Query Language. The streaming segments were then updated in real-time and able to drive next-click experience personalization.

4. Adobe Experience Platform streaming inference with real-time scoring

*Voted most innovative hack*

This hack involved using multiple Adobe Experience Platform services like Pipeline, Pipeline-Smarts and Kafka-Connect to build a streaming inference engine that can enable real-time scoring on streaming data for anomaly detection use cases. The system can support any machine learning model built using any framework like spark, scikit-learn or TensorFlow. It also provides automated runtime artifact generation and model embedding into a streaming context.

The system was designed to power real-time intelligent applications on Adobe Experience Platform. It can also integrate with Data Science Workspace to power real-time scoring for custom machine learning models.

5. Adobe Campaign Classic uses Adobe Experience Platform for segment management

*Voted most market-ready hack*

Adobe Campaign has traditionally offered capabilities to combine profile and segmentation. Customers faced challenges in storing more than 100 million profiles to share segments based on behavioral data flowing from Adobe Analytics and Adobe Audience Manager. Without Adobe Experience Platform it takes roughly 48 hours to ship a segment from Adobe Analytics to Adobe Campaign. This team worked towards reducing time to value for Adobe Campaign Classic. Their application lets audiences for segments be brought into Campaign from Adobe Experience Platform for activation along with profile metadata to be used for personalization of emails.

More Adobe Experience Platform hackathons coming

The 2019 Adobe Experience Platform India Hackathon was a great opportunity to connect with the broader Adobe Experience Platform teams and come up with ideas for new solutions for Adobe Experience Platform customers. It was a very hectic week that demanded our full attention. The first day helped us all understand the process, and the following days were reserved for working on the hack.

Adobe teams in San Jose and Romania had held successful hackathon events previously centered around less complex solutions. We were lucky to be able to leverage their experience to secure our success in planning and executing the first Adobe hackathon designed for Adobe Experience Platform. However, we still went through many challenges. Some we could plan our way around, and others left us forced to improvise on the spot.

We knew very well that Adobe Experience Platform is powerful. We just had no idea that the 2019 Adobe Experience Platform Hackathon would also be an effective tool to connect Adobe engineers all over the world. Our participants were clearly very motivated and eager to solve the customer problems they had identified beforehand. But the interest for this hackathon spread wider than we had expected — all the way to places like San Jose, California. Even though the time difference meant that engineers in this location had to stay up all night to watch the hackathon live, we could see them chatting about each presentation as it was given.

The attendees also showed a great deal of commitment. Most participants stayed very late at the event to talk about interesting problems and solutions. We also received great feedback. One manager told us that he’d liked to have taken more team members along for the hackathon so they could see how useful it was. We will be looking into making events like this one available to a bigger audience in other regions, so our partners can become even more aware of the advantages of Adobe Experience Platform.

Follow the Adobe Experience Platform Community Blog for more developer stories and resources, and check out Adobe Developers on Twitter for the latest news and developer products. Sign up here for future Adobe Experience Platform Meetups.

Resources

1. Adobe Experience Platform — https://www.adobe.com/experience-platform.html
2. Adobe — https://www.adobe.com/index2.html
3. Adobe Primetime — https://www.adobe.com/marketing/primetime.html#
4. Adobe Analytics — https://www.adobe.com/analytics/adobe-analytics.html
5. Nielsen — https://marketingeffectiveness.nielsen.com/our-solutions/campaign-lift/
6. Profile Query Language — https://www.adobe.io/apis/experienceplatform/home/profile-identity-segmentation/profile-identity-seg...
7. Machine learning — https://www.adobe.com/sensei.html
8. Data Science Workspace — https://www.adobe.com/experience-platform/data-science-workspace.html
9. MAGIX — https://www.adobe.io/apis/experienceplatform/home/profile-identity-segmentation/profile-identity-seg...
10. Streaming Segments — https://www.adobe.io/apis/experienceplatform/home/profile-identity-segmentation/profile-identity-seg...
11. Adobe Campaign — https://www.adobe.io/apis/experiencecloud/campaign.html
12. Adobe Audience Manager — https://www.adobe.com/dk/analytics/audience-manager/benefits.html
13. Adobe Campaign — https://www.adobe.io/apis/experiencecloud/campaign.html
14. Adobe Campaign Classic — https://docs.adobe.com/content/help/en/campaign-classic/using/getting-started/starting-with-adobe-ca...

Originally published: Apr 7, 2020