Errors with files when ingesting batch data

[ChanuteJo]

Dear Community,

I have created an ExperienceEvent Schema and tried to ingest data into the dataset. However I have ran into a problem during batch data ingestion. We went back and recheck both the file and the schema and still we were unable to resolve the issue and still get the same error. Image attaches below are the error that we are facing. Please guide us onto how we could resolve this issue. Please let me know if you need anymore information.

Sching

[Harveer_SinghGi1]

Hi @ChanuteJo ,

The error seems to be related to data type mismatch between ingested data and the XDM field(s). This is a generic error though which is generated when there is issue with you data in your batch ingestion but you don't have the error diagnostics enabled for this ingestion, you can read more about it in Retrieving data ingestion error diagnostics documentation.

To get more details on what is happening I'd suggest you attempt the batch ingestion with Error Diagnostics enabled through either UI or API,

{
    "enableErrorDiagnostics": true
}

Once you have this option enabled you can monitor the error details which will provide you exact error on field level.

Cheers!

[ChanuteJo · Accepted solution]

Hi ,

Thankyou for your advice. I went back to look at the data and yes the format mismatched. One of my field in the schema is a DATETIME format but the .csv file is a STRING. I had to use calculated field and the exact DATETIME Format as the .csv file has for it to be able to successfully ingested.

The formula for the created fields I used is below:
date(trim(transactionTimestamp), "yyyy-MM-dd'T'HH:mm:ss.SSSSSS")

Thankyou

[abhinavbalooni]

Hi @ChanuteJo 

Once you've enabled error diagnostics as suggested by @Harveer_SinghGi1 you can try accessing the dataset file using the Data Access API endpoints from here: https://experienceleague.adobe.com/en/docs/experience-platform/ingestion/quality/retrieve-failed-bat...

These should give you a fair idea of what exactly is going wrong with your setup.

Let us know if you still face issues.

Cheers,

Abhinav