Why multiple batch failures at the same time ?

[AtulChavan]

Hi Everyone,
While getting backfill data from Analytics to AEP I have witnessed multiple batch failures had happened at the same time (Ref SS). When I had investigated reason it says  "A batch XXXXXXXXXXXX  is detected as a duplicate". I'm curious to know is it a UI glitch-bug OR under which situation this error pop ups & continues how long ? I can see later (Almost after 38 batches failed) subsequent batches are ingested successfully. Also wondering on what parameter decision happens in single batch number of  records to be ingested.   Thank you.  

[adobechat · Accepted solution]

Has the data been imported manually or is there a system pushing data?

To rule out the basics, have you checked if its not pushing same data multiple times? It could be that it exported 38 times and system rightly processed one of them and marked the rest as duplicates.

[jpetermarias]

This is expected behaviour. Since the ingestion process is asynchronous, this de-duplication functionality is critical to ensure that any duplicate ingestion jobs are not accepted (which would cause event duplication). We have an enhancement request to hide some of these internal details from the end user...but that work is still under review. This will be addressed in future releases.