Does anyone have any example of ingesting a parquet file into AEP? Documentation and Adobe Support says the parquet file must exactly match the XDM schema in AEP. I am have a test file created to do just that. However, there has to be a way to ingest a parquet file and use a mapping set. I prefer to ingest data from client systems without requiring the client to create a specific format just for Adobe.
There must be a way to ingest data from a parquet file that is not XDM compliant. It seems silly to ask a client to reformat / restructure data just for AEP to consume it.
Solved. There is no way to ingest non-xdm compliant parquet. I solved by making the parquet xdm compliant using a data pipeline, ensuring the datetime values are of a format AEP can ingest without errors, and sizing the data files to optimize load time. Parquet file loads 60% faster compared to csv load time.
@davidslaw1 not sure if I've got this right but I assume you are possibly ingesting the data using drag and drop on the dataset UI page, that does not allow for any mapping options.
However most of the Cloud Storage options would let you select the parquet format in your dedicated repository and then go through the 'Mapping' step.
See Map data fields to an XDM schema
Let me know if that helps
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
