what format of parquet file we need to ingest in batches with Platform user interface

[Krishna23]

Hi All,

We are trying to ingest batch data in dataset using parquet file. But, failed to load file data due to field and tenant id missing error.
However, I am able to ingest sample JSON to dataset but failed, when I ingest parquet file after compressing the same sample JSON.
Requesting to suggest XDM compliant parquet file format for our schema. below is the same JSON for schema to load-
{
"_tenantId":{
"country":"Sample value",
"countryCode":"Sample value",
"createdDate":"2018-11-12T20:20:39+00:00",
"creditDays":"Sample value",
"eidmGuid":"Sample value",
"emailId":"Sample value"
},
"_id":"/uri-reference",
"eventMergeId":"Sample value",
"eventType":"advertising.completes",
"identityMap":{
"key":[
{
"authenticatedState":"ambiguous",
"id":"Sample value",
"primary":false
}
]
},
"producedBy":"self",
"timestamp":"2018-11-12T20:20:39+00:00"
}

[adobechat · Accepted solution]

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

[Krishna23]

Yes, I can confirm. As I am loading the sample sample JSON file and it loads correctly. On the other hand, when I am compressing the same sample JSON file and loading, it gets failed.

Could you please suggest the process of generating XDM compliant parquet file format to ingest in AEP dataset.

Thank You
Krishna Gupta