I create a profile schema containing some attributes that are updated with new values using an external process. What I want to do is have the updates in a file (e.g. JSON) with the userID and the columns to update. When I tried this simple approach, the new data does load, but the rest of the profile attributes are no longer there. I was doing this using the Datasets UI to load more data. What is the best way to update profile data from the UI? API is ok as well, it is just harder to demonstrate.
Solved! Go to Solution.
Views
Replies
Total Likes
What I noticed in the past is that you cannot ingest "some" columns only. If you do this, all other parts/properties of the schema will become empty/zero. It doesn't matter if you do this via API or AEP UI. Maybe you run into the same issue?
A resolution maybe to define another schema/dataset with a reduced set of properties and use this to update data. Or you need to get all existing data/columns/properties first, update the required columns and ingest the full set (non-changed and changed properties).
What I noticed in the past is that you cannot ingest "some" columns only. If you do this, all other parts/properties of the schema will become empty/zero. It doesn't matter if you do this via API or AEP UI. Maybe you run into the same issue?
A resolution maybe to define another schema/dataset with a reduced set of properties and use this to update data. Or you need to get all existing data/columns/properties first, update the required columns and ingest the full set (non-changed and changed properties).
Views
Replies
Total Likes
This is to update the original question given the 29 Sept 2021 release. If it works as described it will suffice for updating datasets and, thereby, updating the profile. There are methods defined to set the upsert tag as true on datasets already enabled for profile.
This is a much better solution than using alternate schema and datasets to update records - fewer components, better visibility, and management. Thank you Adobe dev team for enabling this feature
Per 29 SEP 2021 release notes: https://experienceleague.adobe.com/docs/experience-platform/release-notes/2021/september-2021.html?l...
An upsert feature was added to data ingestion: https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/api-overview.html?lang=e...
In that documentation is the following related to batch processing:
“The following is required in order to patch a batch:
Thanks for the hint @DavidSlaw1 .
Do read it correctly? This new feature does not apply to streaming ingestion. For real time updates you still need to prepare a "special" Schema and Dataset.
Update: I just tried. Enabling the dataset with "upsert" ...
"unifiedProfile": [
"enabled:true",
"enabledAt:2022-01-06 09:46:11",
"isUpsert:true"
],
... still has no effect on HTTP ingestion. Fields are overwritten.
there is only one other way that i can think of, but its a bit of coding gymnastics. you could stream the data into a non profile enabled data set and then run a data distiller job (which is kind of batch) to select the data from the profile enabled dataset , match on id, update the new columns and then insert the record into the profile enabled data set. but i agree, just create an enrichment schema if you can and stream it there
Views
Replies
Total Likes
Views
Likes
Replies
Views
Likes
Replies