Deletion of Records from Individual Profile and Custom Class Datasets | Community
Skip to main content
T
trojan_horse
Level 3
October 20, 2025
Solved

Deletion of Records from Individual Profile and Custom Class Datasets

  • October 20, 2025
  • 3 replies
  • 313 views

Hi all,

 

I have an urgent request and would appreciate a quick response.

 

We need to expire or delete old records from both individual profile and custom class datasets in AEP. The records should be removed from both the Profile Store and the Data Lake, and this process should be automated on a schedule - for example, deleting any records older than 30 days.

 

How can this be done in AEP? My understanding is that the data retention policy only works for experience event datasets.

 

Thanks

Best answer by Devyendar

@trojan_horse can you share further info on what is your objective of deleting the data every 30 days? Is it primarily to control profile volume (for license), dataset size or compliance requirement.

 

If it is,

Profile Volume: Instead of deleting, setting up TTL for Experience Events and Pseudonymous profile expiry

Dataset size: Instead of deleting the dataset as suggested by @itsmetechy, deleting Batches as a schedule job from external solution like Lamda through Batches API is better as you don't need to deal with manual dataset creation every time.

Compliance: Privacy Job API can be used

3 replies

itsMeTechy
itsMeTechy
Level 4
October 20, 2025

AEP/RTCDP does not natively support automatically deleting profile records based on their age (e.g., removing records older than 30 days). The platform is designed to maintain comprehensive customer profiles and enable a full 360-degree customer view over time.

 

While there are mechanisms to delete identities from the profile store, these are primarily intended to fulfill GDPR or other regulatory compliance requirements. These options have specific guardrails and limitations and are not suitable for general data lifecycle management.

 

If your use case strictly requires removing records older than 30 days, one practical approach would be to periodically ingest only the current valid profiles into a new dataset and then drop the old dataset. This ensures that only the most recent records remain available in both the data lake and profile store.

    Devyendar
    DevyendarAccepted solution
    Level 6
    October 22, 2025

    @trojan_horse can you share further info on what is your objective of deleting the data every 30 days? Is it primarily to control profile volume (for license), dataset size or compliance requirement.

     

    If it is,

    Profile Volume: Instead of deleting, setting up TTL for Experience Events and Pseudonymous profile expiry

    Dataset size: Instead of deleting the dataset as suggested by @itsmetechy, deleting Batches as a schedule job from external solution like Lamda through Batches API is better as you don't need to deal with manual dataset creation every time.

    Compliance: Privacy Job API can be used

      K
      Karuppiah_Sakthivel
      Level 4
      October 30, 2025

      @trojan_horse 

      In AEP, there’s no native feature to automatically delete or expire profile or custom class records based on age — that capability only exists for Experience Event datasets through the retention policy.

      If the goal is to keep your data fresh or limit the volume of profiles stored, a better long-term approach is to design your data pipeline to handle lifecycle logic before ingestion. For example, filter or exclude records older than 30 days within your source or transformation process so only current data is sent to AEP.

      This avoids unnecessary reprocessing, keeps the Profile Store lean, and prevents dependency on delete operations inside AEP, which are intended mainly for compliance requests. It’s generally cleaner and more reliable to manage this logic upstream rather than within the platform itself.

      Devyendar
      Devyendar
      Level 6
      October 30, 2025

      @karuppiah_sakthivel managing source side filtering is not ideal either, 3 or 6 months down the lane the data ingestion AEP will be older then 30 days and data needs to be managed within AEP, either by deleting batches, applying TTLs to control profile volumes or deleting the entire datasets (although not suggested as it would be a breaking change on the data).

        Terms & ConditionsAccessibility statement

        Loading footer...