AEP Data Lake Retention | Community
Skip to main content
DavidRoss91
Community Advisor
Community Advisor
June 12, 2024
Solved

AEP Data Lake Retention

  • June 12, 2024
  • 1 reply
  • 2308 views

Hi All,

Has anyone had success applying retention to the AEP Data Lake. Not TTL from a dataset perspective, but at the DL level itself. A successful POC for us would be the ability to delete an old batch.

Would the best way, be to use the following: 

https://developer.adobe.com/experience-platform-apis/references/catalog/#operation/listBatches

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by renatoz28

Hi David,

 

You are right. First, you have to list past batches. You would need to order that list by timestamp desc, in order to have the last one and going through the past ones. Then, you could use API services to delete specific batch's. 

 

If you are looking to do it for data cleansing in the data storage, once an Adobe Support Architec told me that this storage only counts for what is in the profile store and not data like. Now the only benefit I'll see is to decrease the processing time when querying a dataset. 

 

Best,

Renato 

1 reply

renatoz28
renatoz28Accepted solution
Level 2
July 8, 2024

Hi David,

 

You are right. First, you have to list past batches. You would need to order that list by timestamp desc, in order to have the last one and going through the past ones. Then, you could use API services to delete specific batch's. 

 

If you are looking to do it for data cleansing in the data storage, once an Adobe Support Architec told me that this storage only counts for what is in the profile store and not data like. Now the only benefit I'll see is to decrease the processing time when querying a dataset. 

 

Best,

Renato 

DavidRoss91
Community Advisor
Community Advisor
September 5, 2024

Hi @renatoz28 

Thank you for the response. Do you have documentation regarding storage in the profile store vs datalake? I wasn't aware of that. 

DavidRoss91
Community Advisor
Community Advisor
September 16, 2024

Hey @brekrut do you happen to know of any documentation on this topic?