_id deduplication functionality for non profile enabled datasets

[TylerKrause]

Hey all! As I believe many of you know the functionality of _id deduplication is specific to Profile store ingestion. We're ingesting events for reporting and analysis off of one source that is profile enabled. We're using another that has additional specific parameters to power the actual transactional communication in AJO. The method of delivery of our source into AEP uses an 'At Least Once' connection so relying on idempotence of that source isn't possible. Outside of having a check for duplicate events in a Journey condition, does anyone have any thoughts on a solution for avoiding duplicate records flowing through the journey?

[bjoern__koth]

Hi @TylerKrause 

different from profiles, events are immutable and cannot be updated.
If you feed in an event with the same _id, to my knowledge, it will still end up in the data lake but the added data wont be available in your profile.

[TylerKrause]

Thanks for the response, Bjoern! 

This definitely reflects what we see.

Our problem is that we rely on the non-profile enabled dataset to trigger off the events and communication (it has additional data and params compared to what we load to profile), and our current experience is multiple actions being triggered off on the same _id. I believe that the product is correctly 'working as intended' but we're hoping to find a solution to this interesting functionality.