Historical data ingestion approach for Profile Attributes | Community
Skip to main content
SahuSa1
SahuSa1
Level 3
April 9, 2026
Question

Historical data ingestion approach for Profile Attributes

  • April 9, 2026
  • 3 replies
  • 49 views

Hi Adobe Community, Greetings!

 

To support a new use case, we need to add few profile type attributes to an existing Individual profile class schema. To ingest the historical data for profile attributes, I know that we can create a new dataset and a new dataflow where we map the Primary Id + New fields so that existing profiles can sync in the values. 

 

However, I wanted to check if we can do the same in the existing dataset without creating a new dataset for this exercise?

And, If we create a new dataflow where we map the Primary Id + New fields and point it to the same/existing dataset - will it overwrite other columns with null values as other columns were not mapped in the dataflow?

 

Thanks,

Sambit

3 replies

deepaks77644073
deepaks77644073
Level 1
April 9, 2026

If we create a new dataflow where we map the Primary Id + New fields and point it to the same/existing dataset - will it overwrite other columns with null values as other columns were not mapped in the dataflow? Yes, can do that but make sure you add ID+ New fields only...

 

SahuSa1
SahuSa1Author
Level 3
April 9, 2026

Hi ​@deepaks77644073 - If I only map the Id + New fields only - in that case with other columns be overwritten with null values?

Thanks,

Sambit

deepaks77644073
deepaks77644073
Level 1
April 9, 2026

In my view, other fields should not be overwritten with null values unless they are explicitly sent as null.

bjoern__koth
Community Advisor and Adobe Champion
bjoern__kothCommunity Advisor and Adobe Champion
Community Advisor and Adobe Champion
April 11, 2026

Hi ​@SahuSa1 

Have you tried enabling your dataset for upsert (API only) and only ingest the newly created fields?

Without upsert enabled, ingesting a record with only a subset of fields will overwrite the entire existing profile record, all fields not present in the new payload are set to null. This is a full replace (PUT semantics).


With upsert enabled (isUpsert: true on the unifiedProfile tag), the behavior changes to a partial update (PATCH semantics): Ingesting only the desired attribute updates that value while all remaining fields retain their current values, no other fields are overwritten with null. 

Cheers from Switzerland!
    SahuSa1
    SahuSa1Author
    Level 3
    April 13, 2026

    Thanks ​@bjoern__koth for you reply. I was checking and I do not have read access when I used Postman to get details of datasets. Could you help me with what access I would require from Admin to be able to enable upsert for the dataset?

     

    Also, when I try to enable the upsert functionality, it is only applied to the desired dataset that is mentioned in the payload. Is my understanding correct?

     

    And, is there anything that I should be aware of before enabling upsert to a dataset? Will there be any adverse effect?

     

    Thanks,

    Sambit

    bjoern__koth
    Community Advisor and Adobe Champion
    bjoern__kothCommunity Advisor and Adobe Champion
    Community Advisor and Adobe Champion
    April 13, 2026

    Hi ​@SahuSa1 

    the upsert enablement only affects the dataset you are applying it to.

    It is a 2-setp process which is described here: https://experienceleague.adobe.com/en/docs/experience-platform/catalog/datasets/enable-upsert


    In terms of permissions, you need View Datasets to read out the current configuration. 

    To be able to modify this, you need (I googled it, but looks alright)

    • Manage Datasets
    • Manage Profiles
    • Manage Schemas
    • Developer access to use the API
    Cheers from Switzerland!
    AmitVishwakarma
    Community Advisor
    AmitVishwakarmaCommunity Advisor
    Community Advisor
    April 11, 2026

    Hi ​@SahuSa1 

    • Same dataset + no upsert: partial payload = full overwrite -> nulls other fields.
    • Same dataset + upsert enabled: partial payload = partial update -> safe to send only ID + new fields.
    • If you can't enable upsert, use a separate profile dataset for the new attributes and rely on merge policy.
    Amit Vishwakarma - Adobe Commerce Champion 2025 | 16x Adobe certified | 4x Adobe SME
      SahuSa1
      SahuSa1Author
      Level 3
      April 13, 2026

      Thanks ​@AmitVishwakarma  for you reply. I was checking and I do not have read access when I used Postman to get details of datasets. Could you help me with what access I would require from Admin to be able to enable upsert for the dataset?

       

      Also, when I try to enable the upsert functionality, it is only applied to the desired dataset that is mentioned in the payload. Is my understanding correct?

       

      And, is there anything that I should be aware of before enabling upsert to a dataset? Will there be any adverse effect?

       

      Thanks,

      Sambit

      Terms & ConditionsAccessibility statement

      Loading footer...