Managing non anonymous profiles - how to remove them

[Michael_Soprano]

Assuming that I have exceeded amount of profiles in CDP (non anonymous). How to manage remove them:

1. There is an endpoint in RTCP to remove profile - may delete multiple profiles at once ?

2 Drop the whole dataset ? 

[bjoern__koth]

Hi @Michael_Soprano 

First of all, I would suggest to reach out to support about this to get their feedback on performance impacts and suggesting the best fit approach, etc.

Else than that, a Privacy Service Delete request should be (please get confirmation as mentioned above) the recommended method for a "full removal" of a specific individual's data, as it targets all data stores and identity links simultaneously since it removes all traces of a Profile from Profile Store, Identity Service and Data Lake and is also capable to process bulk deletion.

This process is specifically designed to handle regulatory data privacy requests (like GDPR's "Right to be forgotten" or CCPA) and ensures the profile data is removed from all three main AEP components:

• Real-Time Customer Profile Store: Removes the profile's unified attribute data.

• Identity Service: Removes the identity graph associations (the link between the authenticated ID, ECID, email, etc.).

• Data Lake: Removes the raw event and record data associated with the profile

Bear in mind though that this process can take days to weeks to fully finish. Once the request is received, 

• the profile data is soft-deleted (not accessible by AEP services) almost immediately.

• the system then processes the full removal across all targeted services. This can take anywhere from 15 to 45 days to complete, depending on the volume and other factors.

[Michael_Soprano]

Thanks @bjoern__koth  for clarification. However if my use-case look like this:

1. I have got Data Mart with identity resolution etc 

2. I ingest data on daily basis

3. Some rules in Data Mart were changed and that would affect how profiles are created in RTCP

I see one solution:

1. Delete dataset - it would be removed in few days 

2. Create new dataset and ingest data from Data Mart 

[Michael_Soprano]

@bjoern__koth  there is new API enpoint. Check this out:

https://experienceleague.adobe.com/en/docs/experience-platform/data-lifecycle/api/workorder

[hegi90]

Hi

in addition what @bjoern__koth already mentioned, there is an advanced data lifecycle management, where you can delete datasets and records (with the data hygiene API you can also automate things). And also check the data retention settings for your dataset. Depending on your use cases you can reduce your retention policy on data lake storage and profile storage. 

Hope that helps. 

Best

Patrick

[KevinFosterDX · Accepted solution]

Just for clarity, if your reason for wanting to delete people data isn't due to an explicit request by the end consumer, the license terms for the Privacy API prohibits its use for this.

The only two allowed options purging obsolete people data are using the Data Lifecycle Record Delete feature or deleting all of the datasets that contain the unwanted data. The first option is normally the one chosen of course.

[marc_stowe_za]

Had a similar problem recently, the customer was 300% over their license limit. we resolved to only keep consent = y for email , and set the rest into a non profile dataset so there was no impact on license.

essentially created a new profile dataset with the required consent records using data distiller (also sourced the consent record via an audience), and then copied all the records into a non profile dataset for CJA to read. then dropped the original.

also, set TTL on AJO tables (if you have AJO) which requires a ticket with Adobe support.

set TTL on web events

there is now an ongoing monthly task to refine the datasets, but will be fine and the licenses are being managed

another approach , is to create a unique namespace for a transient type dataset of profiles for streaming. use this dataset to enrich a marketing dataset with is profile enabled, and uses a different namespace. and then set pseudonymous profile management for the transient namepace, which will remove all records for that namepsace from the sandbox

[arpan-garg]

Hi @Michael_Soprano  - I agree with  KevinFosterDX , if the request to delete data didn't came from the end customers please don't use it. Though it is very convenient to delete data via privacy service API or UI it should not be over used. If you know the identity of the customers that you wan't to delete you can use record delete feature of data lifecycle management to do this.