Ensuring Latest Information Prevails in AEP Streaming Ingestion

Hello there,

We’re encountering a significant issue with data consistency in AEP during streaming ingestion of consent data from a single server-to-server source. Our expectation is that the most recent consent status will replace older information in the CDP/AEP Profile. However, instead, the older consent data is being retained. This data inconsistency is concerning, especially given that reliable, up-to-date data in AEP is crucial for many users.

I’m aware of the upcoming Customer Timestamp Ordering feature which could help resolve this by using source timestamps to prioritize the latest data. However, this feature isn’t available for production yet, has several limitations, and lacks an ETA.

Here’s an example of what we’re seeing:

UserID Status Time

Despite the second record being newer by 13 minutes (based on the source’s wall clock), the profile sometimes ends up with the "Skipped" status instead of the expected "Accepted" status. This inconsistency is problematic as we end up with outdated consent information in the Unified Profile Store.

Given the critical nature of data integrity, what are the best practices or alternative solutions available to address this currently?

Any guidance would be greatly appreciated.

Thank you in advance!