Delete Incorrect Batch File

[filouterrible]

Hi, 

I tried ingesting a batch file using an SFTP connector on a FOLDER path. Unfortunately, the folder contained an unrelated file and the ingestion failed.

The error says: INGEST-1401-400 The row has corrupted data and cannot be read or parsed. Fix the corrupted data and try again.

How can I fix (delete the problematic file) and try again?

I still have 2 files in this batch that are in the right format that I would like to ingest, but I first need to delete the problematic file. I don't understand where these files are kept at the moment. I see them when I look a the failed batches in the API and in the UI, but I don't see how I can fix and try again. 

How can I achieve this? 

Thanks

[BryanRek · Accepted solution]

When ingesting data from the source SFTP connection the data is held within the container which the SFTP connection was mapped to.

See Flow connection for SFTP connection.  This is where the flow path to ingest data from.

https://experienceleague.adobe.com/docs/experience-platform/sources/api-tutorials/create/cloud-stora.... 

[sreeCharan81]

If you are trying to look at the failed batch's dataset files(source). This API might help.

https://developer.adobe.com/experience-platform-apis/references/data-access/#tag/Data-Access/operati...

[filouterrible]

Thanks, but I have already done that.

What I'm trying to figure out is how I can "fix the batch" and try to ingest again. I can see the problematic file, but can't interact with the file or delete the batch altogether.

When trying to delete the batch I get the following error:

There are no revertable predecessor batches for batchId={{BATCH_ID}}

Not sure why this is happening or what to do next? 

[abhinavbalooni]

Hey @filouterrible 

Did you try to enable partial ingestion and check if that helps ?

This would help ensure the entire batch doesn't fail due to the one rogue file.

Let us know if this helps.

Cheers,

Abhinav

[filouterrible]

We did not activate the partial ingestion. This would have solve our problem indeed @abhinavbalooni . 

I was more curious on ways to interact with failed file. 

Where are they kept in the system?

I can see that they failed, but can we access/retrieve them in any way? 

Thanks

[NimashaJain]

@filouterrible Did you find the suggestion solution helpful? It would be great if you can  mark the answer as correct for posterity. If you have found out solution yourself, share it with wider audience in the community.