Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
Bedrock Mission!

Learn more

View all

Sign in to view all badges

what format of parquet file we need to ingest in batches with Platform user interface

Avatar

Level 1

Hi All,

We are trying to ingest batch data in dataset using parquet file. But, failed to load file data due to field and tenant id missing error.
However, I am able to ingest sample JSON to dataset but failed, when I ingest parquet file after compressing the same sample JSON.
Requesting to suggest XDM compliant parquet file format for our schema. below is the same JSON for schema to load-
{
"_tenantId":{
"country":"Sample value",
"countryCode":"Sample value",
"createdDate":"2018-11-12T20:20:39+00:00",
"creditDays":"Sample value",
"eidmGuid":"Sample value",
"emailId":"Sample value"
},
"_id":"/uri-reference",
"eventMergeId":"Sample value",
"eventType":"advertising.completes",
"identityMap":{
"key":[
{
"authenticatedState":"ambiguous",
"id":"Sample value",
"primary":false
}
]
},
"producedBy":"self",
"timestamp":"2018-11-12T20:20:39+00:00"
}

 

2 Replies

Avatar

Level 4

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

Avatar

Level 1

Yes, I can confirm. As I am loading the sample sample JSON file and it loads correctly. On the other hand, when I am compressing the same sample JSON file and loading, it gets failed.

Could you please suggest the process of generating XDM compliant parquet file format to ingest in AEP dataset.

 

Thank You
Krishna Gupta