AEP Dataflow Failing

[Tejasvi_mudumala]

We are receiving data into  (AEP) from multiple sources via Azure Data Factory , using SFTP for file transfers. The files appear successfully in the target location at first, and we have a Dataflow set up in AEP to process them.

However, when the Dataflow is triggered, it fails with an error, and we’ve observed that the source file disappears after the Dataflow running. 

Within our ADF pipeline, there is a step where the files are encrypted and then deleted after processing. While we’re not entirely certain this deletion is directly causing the Dataflow to fail, it seems likely that the file is being removed before the Dataflow has finished reading it.

Error summary CONNECTOR-1001-500: Error in copying the data from Source.

Error code: 2200
Description : /mapBVToAEP_DailyEvents_20250707.jsonl.pgp is not a valid file path

[KumarRishii]

Try below steps to fix this error in AEP Dataflow:

Delay File Deletion in ADF: Add a delay (5–10 minutes) after file transfer before deletion.

Let AEP Move Files Post-Processing: Configure AEP Dataflow to move files to /archive or /error folder after ingestion.

Ensure Correct File Naming: If files are encrypted and renamed (e.g., .pgp), make sure AEP Dataflow is set to pick up the right file name and extension.

Let me know if these steps helps you to resolve the issue.

[venkataak]

Hi Tejasvi, 

Please remove or disable the delete function in the ADF pipeline.or replace the current SFTP-based ingestion with Adobe Data Landing Zone (DLZ) as the primary source. Update the ADF pipeline to move encrypted files directly to DLZ, and configure a new AEP Dataflow using the DLZ connector for ingestion. Since DLZ has a 7-day retention, this ensures reliability and avoids failures due to premature file deletion.

[Sukrity_Wadhwa]

Hi @Tejasvi_mudumala,

Were you able to resolve this query with the help of the provided solutions, or do you still need further assistance? Please let us know. If any of the answers were helpful in moving you closer to a resolution, even partially, we encourage you to mark the one that helped the most as the 'Correct Reply.'

Thank you!