Export AEP Dataset to S3

[sheejo]

Hello All

I am trying to export the AEP datasets to AWS S3. I am able to export some datasets successfully. But when I am trying to export "Profile-Snapshot-Export-XXXX" dataset, I am not able to see in the dataset selection list. Is there any restrictions to export the "Profile-Snapshot-Export-XXXX" dataset?

Thanks in Advance

Sheejo

[birthariyamayank]

Hi,

Dataset Export supports the export of first-generation data, which means raw data directly ingested into AEP from Customer’s data stores or Adobe Experience Cloud solutions.

Though, you can make a copy of such datasets and export it (also, you can automate the process as necessary).

Thanks

[sheejo]

@birthariyamayank , I am not aware of the automation process for copying. Could you please explain in detail? Or any documentation is available?

Thanks in advance

Sheejo

[birthariyamayank]

By automation process, I meant scheduling the query to populate the new dataset with Profile snapshot data on a daily(or need) basis. 

Please refer to this to schedule a query: https://experienceleague.adobe.com/docs/experience-platform/query/ui/query-schedules.html?lang=en

Note: Profile snapshots can be huge data to load on a daily basis, please check contractual limitations and try to optimize when copying it to the new dataset.

[sheejo]

@birthariyamayank , for this, I need to create a dataset same as the "Profile-Snapshot-Export-XXXX" dataset. Do you have any idea how we can create the "Profile-Snapshot-Export-XXXX" dataset copy? Profile-Snapshot-Export-XXXX Schema is not available. If schema is there, we can create. 

[birthariyamayank]

Hi, you can use XDM Individual Profile Schema.

Thanks!

[_Manoj_Kumar_]

Hello @sheejo 

Unfortunately, Only the dataset assigned to out-of-the-box schema types can be exported. 

You will notice that the dataset created by the query service uses "Some Adhoc XDM Schema", which is not compatible with the dataset export feature, and that is why your data set is not appearing in the export list.

[Krish829 · Accepted solution]

Hi Sheejo,

The  "Profile-Snapshot-Export-XXXX" dataset is not available to export to the destinations.

To do so, you have to manually copy the data from the  "Profile-Snapshot-Export-XXXX" dataset to a new profile dataset and set up that new dataset for export to destination. Follow the below steps-

1). Create a new dataset via API(refer below link to create new profile dataset)

https://experienceleague.adobe.com/docs/experience-platform/profile/api/export-jobs.html?lang=en#cre...

2). Initiate export job via API 

https://experienceleague.adobe.com/docs/experience-platform/profile/api/export-jobs.html?lang=en#ini...

3). Next, monitor the above export job status via he API-

https://experienceleague.adobe.com/docs/experience-platform/profile/api/export-jobs.html?lang=en#mon...

Krishna