For streaming ingestion into Adobe Experience Platform using an Individual Profile class schema, which timestamp or field determines how a profile record is ingested and evaluated | Community
Skip to main content
R
ReshmikaPu
Level 2
April 19, 2026
Solved

For streaming ingestion into Adobe Experience Platform using an Individual Profile class schema, which timestamp or field determines how a profile record is ingested and evaluated

  • April 19, 2026
  • 5 replies
  • 89 views

 

Hi Community,​@Asheesh_Pandey  ​@Ankit_Chaudhary ​@NicholeVargas 

I’m looking for some guidance on a streaming ingestion scenario in Adobe Experience Platform.

We have a use case where multiple events are received for a single transaction from an upstream system via streaming ingestion, and the data is ingested into an Individual Profile–enabled dataset. The expectation is that the latest record should be reflected on the profile.

However, the challenge is that multiple events for the same transaction:

  • Arrive within milliseconds of each other
  • Belong to the same streaming batch
  • End up having the same ingestion timestamp

Because of this, AEP often retains only the first record, and the profile is not consistently updated with the latest event.

To address this, we mapped the source system’s published date to the lastUpdatedDate field (OOTB External Source System field group). This approach works when there are up to two events, but in some cases, the source system sends more than four events for the same transaction, and the profile still fails to reflect the latest record—even though the merge policy is timestamp-based.

Adobe Support has indicated that since multiple updates to the same profile fragment occur within milliseconds, there is no guarantee that the latest record will be deterministically applied.

My questions are:

  • Is there a recommended way to handle this scenario within AEP?
  • Are there any best practices around merge policies, timestamp fields, or schema design that could help ensure the latest record consistently wins?
  • Is this an inherent limitation of streaming ingestion, or is there something configurable within AEP to improve reliability in this case?

Any suggestions or insights would be greatly appreciated


From the below screen shot we have same ingestion time but the published date is mapped to external source system filed  will that be able to decide the merge policy ?

the profile should have the data from 1st event which is at 52:40:251 
but in the RTCDP it is showing the one from  52:21:788. 
how can this be handled in AEP ?
is there a way 

https://experienceleague.adobe.com/en/docs/experience-platform/xdm/data-types/external-source-system-audit-attributes

 

This Document Explains that lastUpdateddate can be used for defining the priority but at what point ?
is this true for individual class schema ?
any suggestions would be helpful
Thanks
Reshmika
 

Best answer by Asheesh_Pandey

​@ReshmikaPu This is a classic "race condition" problem with AEP's high-frequency streaming. When two or more events hit the same batch at the same time, the system picks one at random to win.

Steps to deal with this:

  1. Platform allows customers to order by timestamp: This is the big change. Turn on "Customer Timestamp Ordering" in your Merge Policy settings. This tells AEP to not pay attention to the system's ingestion time and instead use your source field to pick the winner.
  2. To make sure this works with events that are sent milliseconds apart, you must have millisecond (or higher) accuracy. If the source system rounds these off to the second, AEP still won't know which one is really last.
  3. You said you were using the ’External Source System Audit Attributes’. This is the perfect place for it. Just make sure that your data prep mapping is changing that source string into the right XDM format.
  4. Adobe Support is technically correct; if everything happens in the same millisecond, it is not deterministic. If the steps above don't work, you might need to look into "debouncing" upstream (putting those four or more events into one final state before sending them) to make sure the profile doesn't get too much.

The Experience League docs on Merge Policies go into detail about how to give your own timestamps more weight than the system's.

5 replies

R
ReshmikaPuAuthor
Level 2
April 20, 2026

​@AmitVishwakarma 
Could you please share your thoughts on the above question?

Michael_Soprano
Michael_Soprano
Level 10
April 21, 2026

Its the XDM Individual Profile or Experience Event? You did not say that explicitly ..

R
ReshmikaPuAuthor
Level 2
April 23, 2026

this is is for XDM profile class

U
UdayKi3
Level 2
April 23, 2026

Hey Reshmika,

What I’d do is to separate raw changes from the current profile state and profile entities are for changable person attributes, while event entities are for actions that happen over time. For transactions like yours that changes in milliseconds, I’d use the XDM ExperienceEvent class schema instead of current Individual Profile class schema.

If you must keep the current Individual profile dataset, then use “streaming upserts via Data Prep” instead of plain repeated creates. use one stable _id for each transaction. do not generate new _id. setup data prep streaming upsert and send later updates with operations.data = merge.

here is the documentation: https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/upserts

Asheesh_Pandey
Community Advisor and Adobe Champion
Asheesh_PandeyCommunity Advisor and Adobe ChampionAccepted solution
Community Advisor and Adobe Champion
April 26, 2026

​@ReshmikaPu This is a classic "race condition" problem with AEP's high-frequency streaming. When two or more events hit the same batch at the same time, the system picks one at random to win.

Steps to deal with this:

  1. Platform allows customers to order by timestamp: This is the big change. Turn on "Customer Timestamp Ordering" in your Merge Policy settings. This tells AEP to not pay attention to the system's ingestion time and instead use your source field to pick the winner.
  2. To make sure this works with events that are sent milliseconds apart, you must have millisecond (or higher) accuracy. If the source system rounds these off to the second, AEP still won't know which one is really last.
  3. You said you were using the ’External Source System Audit Attributes’. This is the perfect place for it. Just make sure that your data prep mapping is changing that source string into the right XDM format.
  4. Adobe Support is technically correct; if everything happens in the same millisecond, it is not deterministic. If the steps above don't work, you might need to look into "debouncing" upstream (putting those four or more events into one final state before sending them) to make sure the profile doesn't get too much.

The Experience League docs on Merge Policies go into detail about how to give your own timestamps more weight than the system's.

- Asheesh
R
ReshmikaPuAuthor
Level 2
April 27, 2026

https://experienceleague.adobe.com/en/docs/experience-platform/profile/customer-timestamp-ordering​@Asheesh_Pandey 
Thank you for the information yes we are checking with adobe on this new feature which is in private beta 

I have a question here if we update the merge policy will that effect the other ingestions 
could you please provide some idea on that 


 

Asheesh_Pandey
Community Advisor and Adobe Champion
Asheesh_PandeyCommunity Advisor and Adobe Champion
Community Advisor and Adobe Champion
April 28, 2026

Yes! especially if this is a default merge policy. While the physical ingestion of data remains unaffected, the unified results (profiles and audiences) built from that data will change. As it may impact the profile and can be a destructive change. Please contact your support to look into the use case and recommend the best option.

 

- Asheesh
    Terms & ConditionsAccessibility statement

    Loading footer...