Content and Commerce AI: Personalizing Your Interactions with Customers Through Content Intelligence

[#1]

Authors: Mihir Naware, Isabel Arguelles, Elliot Sedegah, Arnold Espos, Hyman Chung, Josh van Tonder, and Anne D’Angelo

As a global community, we find ourselves navigating uncharted waters, where we have had to adapt and find new ways to interact with each other exclusively through digital channels. It would seem that we have made a seismic and permanent shift towards embracing the power of digital communication and virtualization. With all these changes, how do marketers evolve? How do brands stay in touch with their customers and build strong relationships, in spite of the physical separation that has become a part of our collective realities?

Adobe recognizes content is the lifeblood of the digital experience. Understanding content and how customers’ interact with it is the key to finding better answers to the quintessential questions marketers grapple with: the “who”, “what”, “when”, “where”, “how”, and “why” of engaging with their customers. In this digital-first world, these questions are more important and relevant than ever before.

Content intelligence may be the answer. It’s a key capability that customers can employ to transform how they work with content. It allows them to form a deep understanding of their customers’ preferences and by doing so, personalize how they interact with each and every one of them, resulting in stronger relationships.

In this article, we outline how we are bringing the power of Adobe’s content intelligence natively into our Digital Experience products, specifically Adobe Experience Manager (AEM), with the creation of a robust extensibility framework which allows the application of AI precisely where it’s needed in the asset workflows. Done right, AI can be transformational — but the key is to get the packaging and features of AI spot-on so that it can best augment the effectiveness of our marketer customers.

What is Content Intelligence?

Content intelligence is a set of Artificial Intelligence (AI) microservices built to understand what aspects of a digital experience resonate with a customer and how the use of those insights could be used to deliver meaningful personalized experiences. For example, we are able to tell you if it was the color, celebrity, hero image or copy that resonated the most with your audience. Adobe’s AI services are built to scale up to enterprise workloads, which means the ability to handle these insights at throughputs of millions of assets and an even higher magnitude of customer interactions.

Figure 1: Content and Commerce AI (beta) capabilities

The outcomes including an ROI that content intelligence delivers to customers are:

• Marketer Productivity: Automate content-related tasks with high accuracy, marketers and site creators are free to scale up their operations, slash delivery times, and focus more on the strategic aspects of their customer engagements.
• Content Velocity: Increase the speed at which content flows from creator to consumer by using AI to provide a robust and rich metadata system.
• Customer Engagement: Deliver increased impactful content and experiences to the customer facilitates better engagement with the content, both in the short-term (in-session or return visits) and longer-term in terms of uplifts in brand loyalty.

How can Content Intelligence be used to improve the experience?

We start out with a core hypothesis: could the deployment of content-intelligence AI microservices in the AEM environment and within workflows. These workflows are currently beloved by marketers and website creators. How can we allow them to be more effective in productivity? Could it help them deliver better value to their customers?

Additionally, how can we develop an AI microservices integration framework with the flexibility to combine AI engines in workflows that are purpose-built for an enterprise's content management needs?

Figure 2: Three phases of experience personalization with content intelligence

The primary content intelligence-related use-cases where we see the largest transformational potential for our AI service is as follows:

• Content Tagging​: Automatically and accurately tag content (both internal and customer-facing) with relevant labels, topics, and concepts for the purposes of organization and delivery​. Accurate organization of content by way of tagging, classification, and categorization is a core capability that greatly enhances your ability to precisely access content and is fundamental to a data-driven approach to understanding KPIs.
• Content Reuse & Authoring​​: Assist content creators and users in the process of adapting existing content efficiently or even in the processing of authoring content for maximum impact. This is a prerequisite for efficient use and re-use of content.
• Metadata Enhancement & Search​: Enhancing search and discoverability with AI-derived metadata can boost the productivity of your remote workforce. Search and Discoverability is fundamental to a highly-functioning organization, especially in the new paradigm in which corporations are decentralized into thousands of micro-offices all across the globe as employees work from home.
• Visual Site Search & Content Similarity:​​ Adding visual search and content similarity for authors, content creators, and marketers, backed with rich metadata is a game-changer when dealing with text and imagery. This allows the use of multiple modes of search, increasing the synergy between textual content and imagery/media.
• Media Transform & Personalization​​: Allowing the adaptation of marketing content to cater to the exact preferences of a particular customer.
• Content-Aware Insights & Personalization​​: Applying the features and metadata associated with assets that form a part of the customer’s experience allows for the discovery of content-aware insights. This enables marketers to understand their customer’s affinities and deliver richer experiences based on customer behavior captured in user profiles.

Our Solution

Content intelligence brings together two technical innovations: AEM as a Cloud Service connected to Content and Commerce AI (beta). It is the first in a deep roadmap of content intelligence AI microservices.

There are four main classes of AI engines within the current beta release of Content and Commerce AI:

• Keyword Extraction — Automatically extract salient keywords and tags from enterprise documents and content fragments​.
• Color Extraction — Automatically label and quantify the color composition of an image. ​
• Visually Similar Content– Deliver visually similar product recommendations to customers, based on intuitive product features like shape, design, and color​.
• Custom Classifiers — Automatically label an enterprise’s documents or images per a corporate taxonomy with custom AI models​.

Using Content and Commerce AI with Adobe Experience Manager

Let’s explore how we could use AEM’s Cloud Service featuring Asset Compute Service to make seamless integration with Content and Commerce AI to help an online retailer of sports equipment serve their customers better and, in doing so, materially impact their KPIs

1. The online e-commerce brand ‘Venzia’ sells a range of sports apparel, from products for indoor athletes who love their exercise bikes, to gear for ski enthusiasts always looking for the perfect equipment for their next downhill run.
2. With Keyword Extraction, Venzia’s marketers were able to go beyond the basic category information they had, and extract product features like material, fit, and performance automatically from product descriptions and up-level those important facets to their customers.
3. With Custom Classifiers, they could understand whether their content was targeted towards Beginners, Hobbyists, or Highly-Technical customers.
4. With Color Extraction, they could accurately label all their product imagery with the right color labels, offloading their asset creation team from having to manually go to each product shot and add in the color. At the same time, the unique color weightage feature helps customers precisely find what they were looking for — for example, a “red jacket” which semantically meant they were looking for a garment that was more than 70% red.
5. And finally, with Visually Similar Content, they could keep their customers engaged in the experience, by showing visually similar product imagery via a recommendation widget on their product description page.

In the longer term, with each click, search, page view, or ‘add to cart’ event, the product details gleaned using the AI services enhanced Venzia’s understanding of their customer’s characteristics and affinities. This is remarkably useful in delivering the best offers or marketing emails to those customers — for example, a subscription to an online class for yoga enthusiasts or a ski holiday package to the highly competent skier.

In broad terms, the Venzia use-cases described above resonate with our customers over and over again, in different situations where AI-backed metadata enrichment and feature extraction can greatly improve workflows in the creation and delivery of content to customers to interact with.

Challenges

When we first started developing our AI microservices strategy we looked at the broader industry to try to understand the gaps in the different offerings. We realized that there were two major challenges that made it hard for companies to get any value from adopting AI:

1. Solutions were not developed for the marketer: A number of different solutions did not prioritize understanding the persona of their target consumer. Designing a product that is best suited for that customer persona is fundamental to maximize the value they capture.
2. Solutions are not native to the customer experience: AI is best applied through seamless infusion into the product experience where the benefits of the technology are made available, but the technical details are hidden to the greatest extent possible.

We took an in-depth look at the use-cases our customers work through with our products every day and the flow of content and metadata through the asset lifecycle. We then stepped back and designed our AI microservices to create outputs that explicitly augment those use-cases and workflows with richer and more accurate metadata. By doing so we have created AI microservices that produce outputs in the language our customers are used to making adoption easier.

We manage this, on the technical side, by developing a bouquet of both pre-trained AI models, such as for Color Extraction and Keyword Extraction that can be readily deployed, and by providing the ability to train and deploy custom engines with limited amounts of data. The services are published on the Sensei AI Framework and are accessible through Adobe’s API gateway — Adobe I/O.

To surmount the second challenge, we wanted to infuse AI natively into the product workflows. We constructed an architecture that allows our customers to pull in AI engines at the appropriate places in their workflows, while largely abstracting away the technical details and allowing flexibility and scalability. In order for this to work, we developed the Asset Compute Service that allows developers to connect to AI engines seamlessly and access the vast array of AI capabilities in a uniform manner.

Figure 3: Asset computing with AEM

Assets Microservices are running in the server-less Adobe I/O Runtime. Previous versions of Adobe Experience Manager have been able to process n/2 Workflows in parallel where n was the number of CPU cores available. With Asset Microservices, this is now completely offloaded and scales automatically. Processing Profiles can be used in Adobe Experience Manager to add custom workers (calling AI microservices from our library of ML/AI offerings) or asset processing through simple and flexible configurations.

Furthermore, Assets Microservices have been designed and built with extensibility in mind. Adobe I/O Runtime allows running 3rd party code with full tenant isolation as part of the asset ingestion process for those cases where simple configuration is not sufficient and the execution of custom code is required. This enables a new spectrum of custom processing with the power of elastic scale.

Figure 4: AEM Cloud Service

Findings

During the development of Content and Commerce AI we have been able to see the value of creating these AI microservices with the marketer in mind. We developed our product thinking of specific marketing use cases that augment their workflows and improve content velocity. During our pilot program, we worked with companies ranging from fashion retailers to news publishers and we were able to validate our core hypotheses. The use of content intelligence in the form of Content and Commerce AI Services, when used to enhance the metadata associated with our customer’s digital assets, allowed for increased worker productivity and boosted content velocity. We also saw a reduction in consulting spend through automation of asset processing and a sharp increase in customer engagement rates.

By using the Asset Compute Service from AEM successfully we addressed huge friction. This allows us to successfully be able to deploy this AI microservices right into the asset creation and processing workflows of our customers and therefore making them native to the customer experience.

Figure 5: Content and Commerce AI Value PropositionWe are running a Beta for AEM (Asset Compute Service) and Content and Commerce AI from August to January 2021. For additional information, please check out our technical documentation here.

Follow the Adobe Experience Platform Community Blog for more developer stories and resources, and check out Adobe Developers on Twitter for the latest news and developer products. Sign up here for future Adobe Experience Platform Meetups.

References

1. Adobe: https://www.adobe.com/
2. Content intelligence: https://www.adobe.com/marketing/experience-manager/content-intelligence.html
3. Digital Experience Products: https://www.adobe.com/experience-cloud.html
4. Adobe Experience Manager (AEM): https://www.adobe.com/marketing/experience-manager.html?promoid=HCS3XPFF&mv=other
5. Asset Compute Service: https://docs.adobe.com/content/help/en/asset-compute/using/introduction.html#:~:text=Asset%20Compute...
6. Asset Microservices: https://www.adobe.com/marketing/experience-manager-assets/asset-microservices.html
7. Adobe I/O: https://www.adobe.io/
8. Content and Commerce AI: https://docs.adobe.com/content/help/en/experience-platform/intelligent-services/content-commerce-ai/...

Originally published: Aug 27, 2020