Skip to main content
Jan 28, 2022 12:58 PM UTC
Solved

what format of parquet file we need to ingest in batches with Platform user interface

Hi All,

We are trying to ingest batch data in dataset using parquet file. But, failed to load file data due to field and tenant id missing error.
However, I am able to ingest sample JSON to dataset but failed, when I ingest parquet file after compressing the same sample JSON.
Requesting to suggest XDM compliant parquet file format for our schema. below is the same JSON for schema to load-
{
"_tenantId":{
"country":"Sample value",
"countryCode":"Sample value",
"createdDate":"2018-11-12T20:20:39+00:00",
"creditDays":"Sample value",
"eidmGuid":"Sample value",
"emailId":"Sample value"
},
"_id":"/uri-reference",
"eventMergeId":"Sample value",
"eventType":"advertising.completes",
"identityMap":{
"key":[
{
"authenticatedState":"ambiguous",
"id":"Sample value",
"primary":false
}
]
},
"producedBy":"self",
"timestamp":"2018-11-12T20:20:39+00:00"
}

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
adobechat
adobechatAccepted solution
Level 5
Jan 30, 2022 12:45 PM UTC

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

2 replies

adobechat
adobechatAccepted solution
Level 5
Jan 30, 2022 12:45 PM UTC

For parquet files, the nesting and field name should match exactly the ones within AEP - Could you check and let me know if this is followed? Because, this could be one of the reason it has failed.

Translate
Krishna23Author
Jan 31, 2022 7:35 AM UTC

Yes, I can confirm. As I am loading the sample sample JSON file and it loads correctly. On the other hand, when I am compressing the same sample JSON file and loading, it gets failed.

Could you please suggest the process of generating XDM compliant parquet file format to ingest in AEP dataset.

 

Thank You
Krishna Gupta

Translate