Errors with files when ingesting batch data | Community
Skip to main content
C
ChanuteJo
Level 3
January 31, 2025
Solved

Errors with files when ingesting batch data

  • January 31, 2025
  • 2 replies
  • 1060 views

Dear Community,

 

I have created an ExperienceEvent Schema and tried to ingest data into the dataset. However I have ran into a problem during batch data ingestion. We went back and recheck both the file and the schema and still we were unable to resolve the issue and still get the same error. Image attaches below are the error that we are facing. Please guide us onto how we could resolve this issue. Please let me know if you need anymore information.

Sching

Best answer by ChanuteJo

Hi ,

 

Thankyou for your advice. I went back to look at the data and yes the format mismatched. One of my field in the schema is a DATETIME format but the .csv file is a STRING. I had to use calculated field and the exact DATETIME Format as the .csv file has for it to be able to successfully ingested.

 

The formula for the created fields I used is below:
date(trim(transactionTimestamp), "yyyy-MM-dd'T'HH:mm:ss.SSSSSS")

 

Thankyou

2 replies

Harveer_SinghGi1
Community Advisor and Adobe Champion
Harveer_SinghGi1Community Advisor and Adobe Champion
Community Advisor and Adobe Champion
January 31, 2025

Hi @chanutejo ,

The error seems to be related to data type mismatch between ingested data and the XDM field(s). This is a generic error though which is generated when there is issue with you data in your batch ingestion but you don't have the error diagnostics enabled for this ingestion, you can read more about it in Retrieving data ingestion error diagnostics documentation.

To get more details on what is happening I'd suggest you attempt the batch ingestion with Error Diagnostics enabled through either UI or API,

 

 

 

 

{ "enableErrorDiagnostics": true }

 

Once you have this option enabled you can monitor the error details which will provide you exact error on field level.

Cheers!

 

 

    C
    ChanuteJoAuthorAccepted solution
    Level 3
    February 5, 2025

    Hi ,

     

    Thankyou for your advice. I went back to look at the data and yes the format mismatched. One of my field in the schema is a DATETIME format but the .csv file is a STRING. I had to use calculated field and the exact DATETIME Format as the .csv file has for it to be able to successfully ingested.

     

    The formula for the created fields I used is below:
    date(trim(transactionTimestamp), "yyyy-MM-dd'T'HH:mm:ss.SSSSSS")

     

    Thankyou

      abhinavbalooni
      Community Advisor
      abhinavbalooniCommunity Advisor
      Community Advisor
      February 4, 2025

      Hi @chanutejo 

       

      Once you've enabled error diagnostics as suggested by @harveer_singhgi1 you can try accessing the dataset file using the Data Access API endpoints from here: https://experienceleague.adobe.com/en/docs/experience-platform/ingestion/quality/retrieve-failed-batches

       

      These should give you a fair idea of what exactly is going wrong with your setup.

       

      Let us know if you still face issues.

       

      Cheers,

      Abhinav

        Terms & ConditionsAccessibility statement

        Loading footer...