Enable a dataset for UPSERT profile using APIs which ingestion will work on this

[BellaSnow5378]

Once the dataset is enabled for Upsert functionality via API, can it also be used for Streaming ingestion?

[brekrut]

Hello @BellaSnow5378 

Yes you can use streaming ingesting after a dataSet has been enabled for upsert, but please be aware there are some limitations and inbound payload changes.

Please refer to this article.

https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/upserts

[BellaSnow5378]

Hi @brekrut 

Problem Statement  :

Once data is ingested through streaming for a particular profile. Ex. (name :xyz , country :spain )in this we need to update the name of an attribute value . For that profile without getting the data override /deleted of other attributes .Is there any way to update one field and retain all the data for that profile  .

Is there any way to do this and if yes can you please let me know the steps .

[BellaSnow5378]

@arpan-garg 

[brekrut]

Hello @BellaSnow5378 

If you are streaming data for into a dataSet which has been enabled for upsert, you will need to use the merge operation as part of the inbound payload.

https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/upserts#payload-with-identi...

When a dataSet is not configured for upsert the last write of data upon the dataset will be reflected upon profile.

[BellaSnow5378]

Hi @brekrut 

Still facing the same issue 

Steps i followed

Created Two dataset

• Profile dataset In this dataset i have enable this for profile and upsert
• Identity dataset In this i have disable the profile and enable the identity service as per doc 
• I have ingested one record through streaming in profile dataset i can see  profile is created 
• when again i have ingested same record with same individual id just to update specific attribute that attribute got updated but rest got deleted 
• i have follow this payload to send the incremental record 
• curl -X POST 'https://dcs.adobedc.net/collection/9aba816d350a69c4abbd283eb5818ec3583275ffce4880ffc482be5a9d810c4b' \
  -H 'Content-Type: application/json' \
  -H 'x-adobe-flow-id: d5262d48-0f47-4949-be6d-795f06933527' \
  -d '{
  "header": {
  "flowId" : "d5262d48-0f47-4949-be6d-795f06933527",
  "imsOrgId": "{ORG_ID}",
  "datasetId": "62259f817f62d71947929a7b",
  "operations": {
  "data": "merge",
  "identity": "create",
  "identityDatasetId": "6254a93b851ecd194b64af9e"
  }
  },
  {
  
   "body": {

          "xdmEntity": {

              "tenantID": {

                  "individualId": "678996061",

                  "patchTestObject": {

                      "country": "singapore"

                  }

              },

              "_id": "/uri-reference",

              "createdByBatchID": "/uri-reference",

              "extSourceSystemAudit": {

                  "createdBy": "Sample value",

                  "createdDate": "2018-11-12T20:20:39+00:00",

                  "externalID": "Sample value",

                  "externalKey": {

                      "sourceID": "Sample value",

                      "sourceInstanceID": "Sample value",

                      "sourceKey": "Sample value",

                      "sourceType": "Sample value"

                  },

                  "lastUpdatedDate": "2024-06-05T05:19:15+00:00"

              }

          }

      }
  
  }
  

[brekrut · Accepted solution]

Hello @BellaSnow5378 

Can you please try this payload for upsert againt the data collection.  Notice the xdmEntity is changed to xdmUpdate

{
"header": {
"msgType": "xdmEntityUpdate",
"msgId": "{{guid}}",
"xactionId": "{{current_timestamp}}{{guid}}",
"createdAt": "{{epoch_time}}",
"originalTimestamp": "{{epoch_time}}",
"datasetId": "<dataSetID>",
"dataStreamId": "",
"imsOrgId": "<IMSORG>",
"xdmSchema": {
"name": "_xdm.context.profile",
"version": "1.0"
},
"source": {
"name": "<sourceName>"
}
},
"body": {
"xdmEntityUpdates": {
"_<tenant>": {
<Attributes are here>

}
}
}
}

[sreeCharan73]

Enabling dataset for Profile, using the API (isUpsertEnabled) is only for enabling upsert on batch ingestion. 

To use the Streaming Upserts, you could either use XDM compatible or non XDM payloads.

Although, your payload looks like XDM payload, for streaming upsert, you could adopt XDM inCompatible ingestion, and follow the https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/upserts#:~:text=this%20exam... and remove the XDM Entity object(in the above) and make sure to have the mapping in the data prep (Dataflow).