Looking for best approach to build composite key in AEP when one field is missing from source data | Community
Skip to main content
R
RS195
Level 2
October 13, 2025
Question

Looking for best approach to build composite key in AEP when one field is missing from source data

  • October 13, 2025
  • 2 replies
  • 4260 views

Hi everyone,

We’re designing schemas in Adobe Experience Platform that use a composite key (for example, Field1 + Field2) for consistency across datasets.

The challenge is that the incoming API data only contains Field1, while another dataset in AEP already holds both Field1 and Field2.
Sending everything through an external system just to attach the missing field would add unnecessary complexity and delay.

What’s the recommended way to handle this kind of scenario directly inside AEP?
Specifically, how can we enrich or complete the composite key during ingestion or processing when one part of the key is available only in another AEP dataset?

Appreciate any insights or examples from others who’ve addressed a similar schema or enrichment setup.

Thanks!

2 replies

Asheesh_Pandey
Community Advisor
Asheesh_PandeyCommunity Advisor
Community Advisor
October 13, 2025

@rs195 I'd suggest either of these option based on your use case

Option 1 : Use a Lookup Schema

Option 2: Use Data Distiller for Join & Enrichment

    R
    RS195Author
    Level 2
    October 14, 2025

    Hi @asheesh_pandey 

    Thanks a lot for the suggestions - this really helps me frame the next step clearly.
    I just want to confirm the correct identity configuration and ingestion sequence for this setup.

    We’re planning to have two schemas:

    • Schema A – Lookup Schema (record type, not profile-enabled)

    • Schema B – Individual Profile Schema (record type, profile-enabled)

    Each schema will contain these fields:

    • Field1: Shared field between both sources (available in both schemas).

    • Field2: Available only in one source (Source 1).

    • Field3: Composite key formed by concatenating Field1 + Field2.

    Source Details:

    • Source 1 (lookup data) includes Field1, Field2, and Field3 (we’ll generate Field3 during mapping using a concat function).

    • Source 2 (API data) includes only Field1.

    Questions / Clarifications:

    1. If we define Field3 as the primary identity in both schemas, does that mean AEP expects this field to always have a value during ingestion?

      • For Source 1 (lookup dataset), Field3 will always be present.

      • But for Source 2 (API source), Field3 won’t exist yet — only Field1 is available.

      Should the Field3 value for Source 2 be generated dynamically at ingestion time through a Data Prep lookup join (using Field1 to retrieve Field2 from the lookup dataset and then concat Field1 + Field2)?
      Or is there a better way to handle this if the primary identity is required?

    2. For Profile enablement:

      • My understanding is that only the Individual Profile Schema should be profile-enabled, and the lookup schema should remain non-profile-enabled, correct?

      • The lookup dataset would then only be used for enrichment during ingestion.

    3. For relationships:

      • Since Field1 exists in both schemas, should we create a one-to-one relationship on Field1 between the Individual Profile schema and the Lookup schema, so the join/enrichment works properly?

    Essentially, the challenge is:

    • The composite identity field (Field3) is required for profile stitching.

    • But one of our sources (API) doesn’t provide it.
      We want to confirm if the correct pattern is to generate or enrich Field3 at ingestion time inside AEP, using the lookup schema as the reference source.

    Would really appreciate your confirmation on the right identity setup and the order of operations here (schema creation, lookup enrichment, identity definition, etc.).

    itsMeTechy
    itsMeTechy
    Level 4
    October 15, 2025

    to answer your questions

     

    question 1: yes. primary identity is always expected. it is mandatory.

    question 2: lookup dataset also needs to be profile enabled. lookup dataset wont be part of profile store though. but through relationship they can be used in audience definition. there are guardrails for lookup dataset to hold only max 5 GB data across all lookup datasets. where did you get the information that we can do lookup while ingesting the data? as far i know it is not possible.

    question 3: depends on relationship of data.

     

    general recommendation is not to use concatenated field as an identity. it is not future proof.

    but my view is still that, you can have field1 and field2 as an independent attribute and mark them as identity in respective schemas. so it will stitch the data together, not sure why this option is  not a suitable option in this scenario.

      itsMeTechy
      itsMeTechy
      Level 4
      October 14, 2025

      am not able to understand your scenario. when you say composite key, do you mean that you want to concatenate two attributes and store it as single attribute and mark it as identity?

      is the schema you are talking about is individual profile class or experience event class?

      if both attributes were not available in a dataset, then you should consider using data distiller.

      R
      RS195Author
      Level 2
      October 14, 2025

      Hi @itsmetechy 

      Thanks for the response - let me clarify.

      Yes, by composite key I mean two attributes (Field1 and Field2) that together form a unique record identifier.
      We’re not concatenating them as a single string manually; both fields are meant to define the identity for profile stitching.

      The schema is built on the Individual Profile class (record type).

      The challenge is that our API source only provides Field1, while another source system provides both Field1 and Field2.
      We want to handle this scenario inside AEP - ideally enriching or combining data from both sources to build the full composite key before writing to the profile-enabled dataset.

      Would appreciate any guidance on the best practice or setup steps for that enrichment within AEP.

      K
      Adobe Employee
      KevinFosterDXAdobe Employee
      Adobe Employee
      October 15, 2025

      Identities must be unique to a single profile.

       

      When you have Field1 and Field2, together do they always uniquely identity a single profile? (I assume that they do. If so then you should concatenate them into a single field marked as an identity either upstream of AEP or during the Data Prep phase of ingestion.)

       

      But what about only Field1? Does every value for Field1 always identity one single profile?

      • If not then this is not a valid identity as you don't really know who this data belongs to. 
      • If so then ignore Field2 and only use Field1 as an identity across both types of data sources.
        Terms & ConditionsAccessibility statement

        Loading footer...