Update Profile Attributes

[DavidSlaw1]

I create a profile schema containing some attributes that are updated with new values using an external process.  What I want to do is have the updates in a file (e.g. JSON) with the userID and the columns to update.  When I tried this simple approach, the new data does load, but the rest of the profile attributes are no longer there.  I was doing this using the Datasets UI to load more data.  What is the best way to update profile data from the UI?  API is ok as well, it is just harder to demonstrate.

[maggod · Accepted solution]

What I noticed in the past is that you cannot ingest "some" columns only. If you do this, all other parts/properties of the schema will become empty/zero. It doesn't matter if you do this via API or AEP UI. Maybe you run into the same issue?

A resolution maybe to define another schema/dataset with a reduced set of properties and use this to update data. Or you need to get all existing data/columns/properties first, update the required columns and ingest the full set (non-changed and changed properties).

[DavidSlaw1]

@maggod Your solution is what I had started working on. That way each update to the subset of columns can simply replace the old record. The Real-Time profile then gets updated with the new values. I am just hoping there is a way to UPDATE a record without having to specify all the attributes and reload the record. Thank you.

[DavidSlaw1]

This is to update the original question given the 29 Sept 2021 release.  If it works as described it will suffice for updating datasets and, thereby, updating the profile.  There are methods defined to set the upsert tag as true on datasets already enabled for profile.

This is a much better solution than using alternate schema and datasets to update records - fewer components, better visibility, and management.  Thank you Adobe dev team for enabling this feature

Per 29 SEP 2021 release notes:  https://experienceleague.adobe.com/docs/experience-platform/release-notes/2021/september-2021.html?l...

An upsert feature was added to data ingestion: https://experienceleague.adobe.com/docs/experience-platform/ingestion/batch/api-overview.html?lang=e...

In that documentation is the following related to batch processing:

“The following is required in order to patch a batch:

• A dataset enabled for Profile and attribute updates. This is done through dataset tags and requires a specific isUpsert:true tag be added to the unifiedProfile array. For details steps showing how to create a dataset or configure an existing dataset for upsert, follow the tutorial for enabling a dataset for Profile updates.
• A Parquet file containing the fields to be patched and identity fields for the Profile. The data format for patching a batch is similar to the regular batch ingestion process. The input required is a Parquet file, and in addition to the fields to be updated, the uploaded data must contain the identity fields in order to match the data in the Profile Store.”

[maggod]

Thanks for the hint @DavidSlaw1 .

Do read it correctly? This new feature does not apply to streaming ingestion. For real time updates you still need to prepare a "special" Schema and Dataset.

Update: I just tried. Enabling the dataset with "upsert" ...

            "unifiedProfile": [
                "enabled:true",
                "enabledAt:2022-01-06 09:46:11",
                "isUpsert:true"
            ],

... still has no effect on HTTP ingestion. Fields are overwritten.

[marc_stowe_za]

there is only one other way that i can think of, but its a bit of coding gymnastics. you could stream the data into a non profile enabled data set and then run a data distiller job (which is kind of batch) to select the data from the profile enabled dataset , match on id, update the new columns and then insert the record into the profile enabled data set. but i agree, just create an enrichment schema if you can and stream it there