what format of parquet file we need to ingest in batches with Platform user interface | Community
Skip to main content
January 28, 2022
Solved

what format of parquet file we need to ingest in batches with Platform user interface

  • January 28, 2022
  • 1 reply
  • 2081 views

Hi All,

We are trying to ingest batch data in dataset using parquet file. But, failed to load file data due to field and tenant id missing error.
However, I am able to ingest sample JSON to dataset but failed, when I ingest parquet file after compressing the same sample JSON.
Requesting to suggest XDM compliant parquet file format for our schema. below is the same JSON for schema to load-
{
"_tenantId":{
"country":"Sample value",
"countryCode":"Sample value",
"createdDate":"2018-11-12T20:20:39+00:00",
"creditDays":"Sample value",
"eidmGuid":"Sample value",
"emailId":"Sample value"
},
"_id":"/uri-reference",
"eventMergeId":"Sample value",
"eventType":"advertising.completes",
"identityMap":{
"key":[
{
"authenticatedState":"ambiguous",
"id":"Sample value",
"primary":false
}
]
},
"producedBy":"self",
"timestamp":"2018-11-12T20:20:39+00:00"
}

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by adobechat

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

1 reply

adobechat
adobechatAccepted solution
Level 5
January 30, 2022

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

Krishna23Author
January 31, 2022

Yes, I can confirm. As I am loading the sample sample JSON file and it loads correctly. On the other hand, when I am compressing the same sample JSON file and loading, it gets failed.

Could you please suggest the process of generating XDM compliant parquet file format to ingest in AEP dataset.

 

Thank You
Krishna Gupta