Validation of Source Data via HTTP API Source Connector

[RyanMoravick]

Hello all,

I am trying to validate incoming source data into the HTTP API Source Connector in Adobe CDP and getting some errors I have been unable to resolve. 

My initial question is: If I am trying to ingest this data as an event into Adobe CDP, does my source file JSON need to include _id and timestamp? Do these fields need to be included in the payload coming from our backend systems? The error I am receiving is saying that required key (_id) is not found?

Thank you,

[DavidRoss91]

Hi @RyanMoravick 

Event data does need to include a unique _id value for ingestion

[RyanMoravick]

@DavidRoss91 do you know if the HTTP API Source Connector is able to handle an array in the ingestion payload? When I test this using postman I am able to successfully ingest a record, but when pushing a large payload from Databricks, im getting ingestion errors about _id being missing.

[RyanMoravick]

@DavidRoss91 something like this for example

{
"_object": {
"LID": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx",
"AsOf": "2024-11-05",
"rfCategory": "Loyal",
"rfMonetaryCategory": "Other"
},
"_id":"2024-11-05Loyalxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxOther",
"eventType":"rfm.status.update",
"timestamp":"2024-11-05T00:00:00.000Z"
}