Issue with Streaming Upsert for Object Arrays in AEP – Trying to Append Instead of Overwrite

[SiddarthK]

Hi everyone,

I’m currently working on implementing streaming upserts in Adobe Experience Platform (AEP) for an object array field in my schema. Both the schema and the associated dataset are enabled for Profile, and I’ve configured the HTTP API connection accordingly.

In the Data Prep mapping, I’m using the expression:

upsert_array_append(<tenantId>.hobbies3)

to map the incoming hobbies3 array. My intention is to append new hobby objects to the existing array, rather than overwriting it entirely on each update.

I’ve also ensured that:

• Upsert is enabled on the dataset, as described in the official documentation [link].
• The request headers and payload follow the format and requirements outlined in the streaming upsert guide [link].

Despite following the documentation and configuration steps, the array is still being overwritten instead of appended in the profile. I’m specifically working with an array of objects, not simple strings or primitives.

Has anyone successfully implemented upsert_array_append with object arrays in streaming upserts? Any guidance or insights on what might be going wrong would be greatly appreciated.

Thanks in advance!

[Michael_Soprano]

Upsert should be used rather in an one-off use-cases?

[KevinFosterDX]

Arrays and upserts (partial record updates) haven't been a good mix. I've never seen anyone be happy with the results even after extensive attempts.

My recommend is to break out those arrays to a separate schema/dataset where you don't need to use upsert.  Replace the full array each time any potion changes. Let the merge policy pull those updated arrays into the full profile at the time of segmentation.