Is it possible to stream and batch ingest data simultaneously from different sources into the same profile dataset in Adobe Experience Platform (AEP)?

Question

I’m curious about the technical feasibility and best practices. Is combining streaming and batch ingestion into a single profile dataset advisable? What are the key considerations or potential pitfalls to be aware of when setting this up?

Thanks in advance for any insights!

Asheesh_Pandey · Accepted Answer

@yatinsh It is technically feasible to stream and batch ingest data simultaneously into the same profile dataset. Key Considerations: Define identity strategy, XDM schema, and source mappings. Test in sandbox to identify and resolve issues like schema mismatch, data duplication, conflict merge rules, profile fragments etc.

Parvesh_Parmar · Answer

Hi @yatinsh ,As @Asheesh_Pandey mentioned, yes — it is technically possible.However, from a future-proofing and best practices perspective, it's not the ideal approach. If there's an issue with one data source, it can become difficult to debug or analyze the root cause.Also, if incorrect data is sent from one source and you need to delete it, there's a risk of having to clean the entire dataset — which could result in loss of data from both sources.A better solution would be to create two separate datasets, one for each source. You can then stitch the data together based on identity namespaces and control how the data is prioritized using merge policies.Hope this helps!Kind regards,Parvesh

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded