Richer Data, Smarter Journeys : Why Dataset Lookup is the key to next-level personalization in AJO

[#1]

Holiday campaigns, tracking abandoned carts, and real-time promotions create immense pressure for marketing teams. Many retailers and service brands face a familiar challenge: enriching customer journeys with relevant data across multiple touchpoints, channels, and customer segments—without hitting system limits or causing delays.

Before Dataset Lookup Node with ELS, this process relied on Profile Access-x with Cosmos DB, which often created bottlenecks:

• Limited throughput (4,000 RUs per tenant/org/sandbox)

• High risk of HTTP 429 errors under heavy load

• Complex schema requirements for profile-linked data

• Limited ability to leverage non-profile datasets like product catalogs or reservations

Let’s explore how the Dataset Lookup Node powered by Entity Lookup Service (ELS) solves these challenges, delivering faster, more flexible, and scalable journey enrichment.

The Challenge: Profile Access-x Limitations

Bottlenecks in the Old Approach

Profile Access-x was designed to fetch data from the Unified Profile Store (Cosmos DB). While adequate for small-scale lookups, the design does not scale for high-volume, concurrent journeys.

• Throughput Ceiling: Cosmos DB enforces a per-tenant/org/sandbox Request Unit (RU) limit (~4,000 RUs). Under concurrent journey executions, read operations rapidly exhaust available RUs.

• Rate Limiting: Once the quota is exceeded, Cosmos DB returns HTTP 429 (Too Many Requests) responses. This introduces latency and causes enrichment steps in the journey to fail or retry.

• Data Modeling Constraint: Profile Access-x is optimized for profile-linked entities. Attempting to enrich journeys with non-profile datasets (e.g., catalogs, reservations, instructional content) requires schema contortions.

Some implementations attempted to embed thousands of product or inventory records inside a single profile object to bypass dataset limitations. This approach creates several technical issues:

1. Hot Partitioning: Large payload reads on a single profile create partition hotspots in Cosmos DB.

2. RU Spikes: Bulk read/write on oversized profile documents drastically increases RU consumption, accelerating quota exhaustion.

3. Journey Instability: Under load testing, concurrent lookups on these oversized profiles consistently triggered 429 errors, leading to partial or failed journey executions.

Net Result: Profile Access-x introduces scaling bottlenecks that prevent reliable high-volume personalization and limit the ability to leverage broader datasets.

Why Dataset Lookup Node Was Introduced : Key capabilities

Marketing teams needed:

• Access to broader datasets—product catalogs, store locations, reservations

• Flexible enrichment without profile enablement

• Faster journey configuration with fewer bottlenecks

Solution: Dataset Lookup Node + Entity Lookup Service (ELS)

• Provides direct access to record-based datasets in Adobe Experience Platform (still Cosmos-backed but optimized for journey enrichment)

• Supports dynamic personalization with Expression Editor

• The newer @datasetLookup{} function was introduced so that the 'lookup-up' data can be used in Condition and Custom Action nodes.

• Eliminates dependency on Profile Access-x, avoiding 429 errors

Results: The enriched data is stored in journey context, enabling downstream use in conditions, custom actions, or personalized messages.

Resolving common use cases

1. Category Spend Segmentation (Baby, Garden, Furniture)

• Challenge (Old Approach): Profile Access-x couldn’t efficiently aggregate cart spend by category without hitting Cosmos DB throughput limits.

• Resolution (Dataset Lookup):

  • Cart items written to event

  • Lookup categories via @datasetLookup{}

  • Aggregate spend per category in-journey

  • Trigger coupon delivery if spend > $100

• Impact: Creates new-to-category segments with precise targeting, enabling cross-sell and upsell opportunities.

2. Product-Linked Instructions (Recipes, Plant Care, Furniture Setup)

• Challenge (Old Approach): Instructional datasets (recipes, care tips) weren’t profile-enabled, blocking personalization.

• Resolution (Dataset Lookup):

  • Purchase SKU matched against “Instructional Content” dataset

  • Journey enriched with matched instructions (recipe card, care guide, setup manual)

  • Personalized email triggered automatically

• Impact: Provides contextual post-purchase engagement, building trust and encouraging related purchases.

3. Abandoned Cart Recovery

• Challenge (Old Approach): Profile Access-x made it complex to attach detailed SKU-level info (image, price, description) in abandoned cart emails.

• Resolution (Dataset Lookup):

  • “Cart Abandoned” event triggers journey

  • Enrich event with SKU attributes (product name, image, price) via Dataset Lookup

  • Populate email dynamically with enriched cart details

• Impact: Increases cart recovery rates with personalized reminders rather than generic nudges.

Dataset Lookup Activity Specifications

It is an Orchestration Activity that enables journeys to dynamically fetch data from Adobe Experience Platform (AEP) datasets during runtime. This allows for real-time enrichment and expression evaluation using data that may not reside in the profile or event payload. 

It requires the following inputs: 

• Select a dataset from AEP to perform lookups at runtime (schema field is auto populated) - It should be enabled by Entity Lookup Service (ELS enabled)

• Define lookup keys using journey context expressions 

• Choose fields to retrieve from the dataset 

In the Advanced Mode Editor in Journey Expressions, you can reference the results of a dataset lookup activity using the "datasetLookup" syntax as follows: 

@datasetLookup{MyDatasetLookUpActivity1.entities}  

In the Personalization Editor, you can reference the results of a dataset lookup activity using a datasetLookup contextual path as follows:

{{context.journey.datasetLookup.1482319411.entities}}

Results: Potential Transformation

Efficiency Gains

• Reduce bottlenecks caused by 429 errors

• Enrich journeys with real-world datasets in minutes instead of days

• Minimize setup complexity with fewer schema restrictions

Marketing Impact

• Launch campaigns faster with dynamic, data-driven enrichment

• Personalize at scale across multiple customer segments

• Test and iterate quickly using enriched dataset parameters

Key Takeaways

• Scale Personalization Without Complexity: Enrich journeys using ELS-enabled datasets without being constrained by profile store limits

• Real-Time Adaptability: Respond instantly to inventory, catalog, or loyalty updates without redesigning journeys

• Faster Time-to-Market: Reduce journey setup time while improving personalization depth

• Enhanced Testing Capabilities: Test multiple conditions and variations in real time

Dataset Lookup Node with ELS transforms AJO journeys, enabling smarter, faster, and more reliable personalization—turning bottlenecks into business opportunities.