Skip to main content
SahuSa1
SahuSa1
Level 3
April 9, 2026
Solved

Historical data ingestion approach for Profile Attributes

Hi Adobe Community, Greetings!

 

To support a new use case, we need to add few profile type attributes to an existing Individual profile class schema. To ingest the historical data for profile attributes, I know that we can create a new dataset and a new dataflow where we map the Primary Id + New fields so that existing profiles can sync in the values. 

 

However, I wanted to check if we can do the same in the existing dataset without creating a new dataset for this exercise?

And, If we create a new dataflow where we map the Primary Id + New fields and point it to the same/existing dataset - will it overwrite other columns with null values as other columns were not mapped in the dataflow?

 

Thanks,

Sambit

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
bjoern-koth
bjoern-kothAccepted solution
Adobe Employee
April 11, 2026

Hi ​@SahuSa1 

Have you tried enabling your dataset for upsert (API only) and only ingest the newly created fields?

Without upsert enabled, ingesting a record with only a subset of fields will overwrite the entire existing profile record, all fields not present in the new payload are set to null. This is a full replace (PUT semantics).


With upsert enabled (isUpsert: true on the unifiedProfile tag), the behavior changes to a partial update (PATCH semantics): Ingesting only the desired attribute updates that value while all remaining fields retain their current values, no other fields are overwritten with null. 

Cheers from Switzerland!
View in thread

10 replies

deepaks77644073
deepaks77644073
Level 1
April 9, 2026

If we create a new dataflow where we map the Primary Id + New fields and point it to the same/existing dataset - will it overwrite other columns with null values as other columns were not mapped in the dataflow? Yes, can do that but make sure you add ID+ New fields only...

 

Translate
SahuSa1
SahuSa1Author
Level 3
April 9, 2026

Hi ​@deepaks77644073 - If I only map the Id + New fields only - in that case with other columns be overwritten with null values?

Thanks,

Sambit

Translate
deepaks77644073
deepaks77644073
Level 1
April 9, 2026

In my view, other fields should not be overwritten with null values unless they are explicitly sent as null.

Translate
bjoern-koth
Adobe Employee
bjoern-kothAdobe EmployeeAccepted solution
Adobe Employee
April 11, 2026

Hi ​@SahuSa1 

Have you tried enabling your dataset for upsert (API only) and only ingest the newly created fields?

Without upsert enabled, ingesting a record with only a subset of fields will overwrite the entire existing profile record, all fields not present in the new payload are set to null. This is a full replace (PUT semantics).


With upsert enabled (isUpsert: true on the unifiedProfile tag), the behavior changes to a partial update (PATCH semantics): Ingesting only the desired attribute updates that value while all remaining fields retain their current values, no other fields are overwritten with null. 

Translate
Cheers from Switzerland!
    SahuSa1
    SahuSa1Author
    Level 3
    April 13, 2026

    Thanks ​@bjoern__koth for you reply. I was checking and I do not have read access when I used Postman to get details of datasets. Could you help me with what access I would require from Admin to be able to enable upsert for the dataset?

     

    Also, when I try to enable the upsert functionality, it is only applied to the desired dataset that is mentioned in the payload. Is my understanding correct?

     

    And, is there anything that I should be aware of before enabling upsert to a dataset? Will there be any adverse effect?

     

    Thanks,

    Sambit

    Translate
    bjoern-koth
    Adobe Employee
    bjoern-kothAdobe Employee
    Adobe Employee
    April 13, 2026

    Hi ​@SahuSa1 

    the upsert enablement only affects the dataset you are applying it to.

    It is a 2-setp process which is described here: https://experienceleague.adobe.com/en/docs/experience-platform/catalog/datasets/enable-upsert


    In terms of permissions, you need View Datasets to read out the current configuration. 

    To be able to modify this, you need (I googled it, but looks alright)

    • Manage Datasets
    • Manage Profiles
    • Manage Schemas
    • Developer access to use the API
    Translate
    Cheers from Switzerland!
      AmitVishwakarma
      Community Advisor
      AmitVishwakarmaCommunity Advisor
      Community Advisor
      April 11, 2026

      Hi ​@SahuSa1 

      • Same dataset + no upsert: partial payload = full overwrite -> nulls other fields.
      • Same dataset + upsert enabled: partial payload = partial update -> safe to send only ID + new fields.
      • If you can't enable upsert, use a separate profile dataset for the new attributes and rely on merge policy.
      Translate
      Amit Vishwakarma - Adobe Commerce Champion 2025 | 17x Adobe certified | 6x Adobe SME
        SahuSa1
        SahuSa1Author
        Level 3
        April 13, 2026

        Thanks ​@AmitVishwakarma  for you reply. I was checking and I do not have read access when I used Postman to get details of datasets. Could you help me with what access I would require from Admin to be able to enable upsert for the dataset?

         

        Also, when I try to enable the upsert functionality, it is only applied to the desired dataset that is mentioned in the payload. Is my understanding correct?

         

        And, is there anything that I should be aware of before enabling upsert to a dataset? Will there be any adverse effect?

         

        Thanks,

        Sambit

        Translate
        Terms & ConditionsAccessibility statement

        Loading footer...