Maximizing Customer Journey Analytics Data Value with Adobe Experience Platform Transformations

[#1]

Introduction

The Adobe Experience Platform (AEP) provides different services and product options for manipulating data at various stages of ingestion and processing associated with Customer Journey Analytics (CJA). Transformations can be applied at multiple points to enhance the effectiveness and interoperability of analytic datasets, which can improve the final throughput to Customer Journey Analytics.

One key area for early data transformation is in the Data Prep (Mapper) stage. Here, you can create calculated and transformed fields, as well as map fields from the source data to the destination Experience Data Model (XDM). By using the Data Prep stage, you can transform data into a validated state that is more useful and relevant to business use cases and needs.

Data transformations can also be applied in the post-ingestion stage using Data Distiller. This allows for the creation of new Query Service batch-processed datasets. By cleaning, shaping, and manipulating data, you can form new composed datasets in the Experience Platform Data Lake.

Transformations can also be applied to Customer Journey Analytics within the CJA Derived Fields and CJA Component Settings sections, in addition to the previously mentioned areas. These options provide retroactive flexibility to customize data and meet specific CJA criteria, producing outputs that serve as final end dimensions and metric components.

The Adobe Experience Platform provides a variety of tools for manipulating data across the entire data value chain. This allows businesses to maximize their datasets and gain insights in CJA that can drive growth and success.

Data Lifecycle to Customer Journey Analytics

To begin, it is important to have a comprehensive understanding of the potential phases involved in the data lifecycle for ingesting and transforming data in the Adobe Experience Platform. This process can be complex and nuanced, with many different points where transformations can be applied in order to achieve the desired outcomes.

The diagrams below provide a visual representation of the various possible phases involved in the data lifecycle for AEP. It is essential to understand each of these phases and how they relate to one another in order to make informed decisions about data ingestion and transformation.

 

Once you have a thorough understanding of the data lifecycle in Adobe Experience Platform (AEP), you can make informed decisions about how to approach data transformations for Customer Journey Analytics (CJA). This involves carefully considering the various functional options available for the stages of data ingestion and transformation, and understanding the broader implications of each choice.

 

Using the matrix outlined later in this blog post, you can tailor your approach to meet the specific needs of your organization and data. This will enable you to make better-informed decisions regarding when and how to perform transformations, ultimately resulting in a more effective and efficient data management strategy for your organization.

AEP/CJA Data Manipulation Matrix

Understanding the sequence, timing, and means of data manipulation throughout the data lifecycle, as well as the trade-offs of different Adobe Experience Platform options, is crucial. The following matrix as of June 2023 provides a high-level comparison of data manipulation and transformation options available to your organization within the Adobe Experience Cloud. By understanding the underlying solution, the phase, and level of semantic data processing, you can make informed decisions about where to apply transformations, depending on the transitory or settled state of the data.

AEP/CJA Data Manipulation Solution	Data Prep	Data Distiller	CJA Derived Fields	CJA Data View Component Settings
Solution Functionality	Data Prep (Mapper) is a option to transform data to XDM via validation, enrichment, hashing, string filtering, and special operations. Data Prep functions can be used to compute and calculate values based on what is entered in source fields. • String functions • Regex functions • Hashing functions • URL functions • Date and time functions • Hierarchy functions (Objects, Arrays) • Logical Operators • Aggregation • Type conversion • JSON functions • Special operations • User agent functions • Object copy	Execute batch queries in Query Service to perform post-ingestion data preparation using supported ANSI SQL, Adobe defined-SQL functions, and Spark SQL functions encompassing: • Math and statistical operators and functions • Logical operators and functions • Date/time functions • Arrays • Datatype casting functions • Conversion and formatting functions • Data evaluation • Current information • Higher order functions	Data manipulations via rule-builder interface at the Connections level to retroactively establish derived field components (metric or dimension) using the following functions: • Concatenate • Case When • Find and Replace • Lookup • URL Parse	Granular components have settings that are retroactive and non-destructively set at the individual Data View level. The selected core settings involving with data manipulation are as follows: • Format (Metric Applicable Only) • Set substring (Dimension Applicable Only) • No value options (Dimension Applicable Only) • Include exclude values
Use Cases	During data ingestion, mapping and data manipulation are performed to prepare for either batch or streaming ingestion. One specific use case is semantic enrichment, which can resolve schema differences between Adobe Analytics report suites or disparate datasets.	Generate insightful derived datasets by writing simple-to-complex batch processing queries for data enrichment, cleansing, shaping, and manipulation for the data landed in the Adobe Experience Platform. Make your data ready for platform-based app consumption.	• Marketing Channels via Function template • Targeted Data Refinement, Clean Up, & Removal • Data Expansion • Small Scale Lookups	• Component-level data parsing, clean up, and include/exclude logic • No value logic handling
Limits/Guardrails	This is limited to the available Data Prep functions, as outlined in the Data Prep List of functions. Only row-level transformations are supported; it is not possible to perform any joins, aggregations, or lookups.	Performance considerations and soft/hard limits as outlined in Guardrails for Query Service.	Only fields in Event datasets can be used at this time. Profile and Lookup fields support is planned. Functions have varying limits per derived field as denoted in the Function reference details. The following limitations apply to the derived field functionality in general: -You can use a maximum of 10 different schema fields (not including standard fields) when defining rules for a derived field. -You can have a maximum of 100 derived fields per CJA connection.	Limited to available per component settings per Data View as outlined in Component settings overview.
Impact Scope	Data Prep allows data engineers to map, transform, and validate data to and from Experience Data Model (XDM). Data Prep is a “Map” step in the Data Ingestion processes before landing in the Experience Platform Data Lake.	Batch processing queries are managed, monitored, and operationalized by the users to clean, shape, manipulate, and enrich datasets within the Adobe Experience Platform.	Derived fields are managed at a Connection level in CJA. Any change made to a derived field in any of the Data views associated with that Connection applies across all associated Data views.	Component settings are managed per individual component within a given CJA Data View. All changes are reflected only within the specific involved Data View.

Conclusion

After reviewing the available options for each stage of data ingestion and transformation, organizations should consider the broader implications of their choices. This involves analyzing how each option affects the entire data value chain, from ingestion to transformation and beyond. By doing so, organizations can make the best decisions possible about their approach to data transformations for Customer Journey Analytics.

In addition, organizations should evaluate how these decisions effect their overall data strategy, including governance and data quality. Taking the time to fully understand the different phases and workloads involved in the data lifecycle will enable companies to not only leverage the full capabilities of AEP but also identify areas for improvement and optimization. Analyzing the implications of each option can help companies make more informed decisions and ensure they are fully utilizing the capabilities of the Adobe Experience Platform to generate valuable insights in Customer Journey Analytics.