Delete Incorrect Batch File | Community
Skip to main content
F
filouterrible
Level 2
December 18, 2023
Solved

Delete Incorrect Batch File

  • December 18, 2023
  • 3 replies
  • 2248 views

Hi, 

I tried ingesting a batch file using an SFTP connector on a FOLDER path. Unfortunately, the folder contained an unrelated file and the ingestion failed.

 

The error says: INGEST-1401-400 The row has corrupted data and cannot be read or parsed. Fix the corrupted data and try again.

 

How can I fix (delete the problematic file) and try again?

 

I still have 2 files in this batch that are in the right format that I would like to ingest, but I first need to delete the problematic file. I don't understand where these files are kept at the moment. I see them when I look a the failed batches in the API and in the UI, but I don't see how I can fix and try again. 

How can I achieve this? 

Thanks

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by BryanRek

    When ingesting data from the source SFTP connection the data is held within the container which the SFTP connection was mapped to.

     

    See Flow connection for SFTP connection.  This is where the flow path to ingest data from.

    https://experienceleague.adobe.com/docs/experience-platform/sources/api-tutorials/create/cloud-storage/sftp.html?lang=en. 

    3 replies

    BryanRek
    Community Advisor
    BryanRekCommunity AdvisorAccepted solution
    Community Advisor
    December 19, 2023

    When ingesting data from the source SFTP connection the data is held within the container which the SFTP connection was mapped to.

     

    See Flow connection for SFTP connection.  This is where the flow path to ingest data from.

    https://experienceleague.adobe.com/docs/experience-platform/sources/api-tutorials/create/cloud-storage/sftp.html?lang=en. 

      S
      sreeCharan81
      Level 2
      December 21, 2023

      If you are trying to look at the failed batch's dataset files(source). This API might help.

      https://developer.adobe.com/experience-platform-apis/references/data-access/#tag/Data-Access/operation/retrieveFailedBatch

       

        F
        filouterribleAuthor
        Level 2
        December 21, 2023

        Thanks, but I have already done that.

        What I'm trying to figure out is how I can "fix the batch" and try to ingest again. I can see the problematic file, but can't interact with the file or delete the batch altogether.

         

        When trying to delete the batch I get the following error:

        There are no revertable predecessor batches for batchId={{BATCH_ID}}

         

        Not sure why this is happening or what to do next? 

         

          F
          filouterribleAuthor
          Level 2
          January 8, 2024

          Hey @filouterrible 

          Did you try to enable partial ingestion and check if that helps ?

           

          This would help ensure the entire batch doesn't fail due to the one rogue file.

           

          Let us know if this helps.

           

          Cheers,

          Abhinav


          We did not activate the partial ingestion. This would have solve our problem indeed @abhinavbalooni . 

           

          I was more curious on ways to interact with failed file. 

          Where are they kept in the system?

          I can see that they failed, but can we access/retrieve them in any way? 

           

          Thanks

            NimashaJain
            Adobe Employee
            NimashaJainAdobe Employee
            Adobe Employee
            January 8, 2024

            @filouterrible Did you find the suggestion solution helpful? It would be great if you can  mark the answer as correct for posterity. If you have found out solution yourself, share it with wider audience in the community.

            Terms & ConditionsAccessibility statement

            Loading footer...