Delete profiles from AEP

[sheejo]

Hello

We have a requirement to delete few profiles from AEP. What are the ways we can delete the profiles from AEP? If yes, how many profiles we can delete?

Please help.

Thanks in advance

Sheejo Rapheal

[arijitg]

Depending on your requirement you can delete them by entire dataset or by a batch or even by individual profiles based on namespace.

[sheejo]

Thanks for your reply. How can we do the deletion? Is there any API for that or any other process?

[arijitg]

I can answer better if you can tell me how do you want to delete them.

By Dataset/Batch/Namespace?

[sheejo]

We have 7K+ profiles to delete. So, we will go with Batch.

[arijitg]

If you want to delete them based on Batch ID you can refer bellow : Batch Ingestion API Guide | Adobe Experience Platform

Delete a batch

A batch can be deleted by performing the following POST request with the action=REVERT query parameter to the ID of the batch you wish to delete. The batch is the marked as “inactive”, making it eligible for garbage collection. The batch will be asynchronously collected, at which time it will be marked as “deleted”.

API format

POST /batches/{BATCH_ID}?action=REVERT

[maggod]

I don't think batch deletion API will solve this request. Deleting a batch doesn't mean deleting the profile.

I understand @sheejo wants to delete 7000 individual profiles (not batches, profiles are known by their ID) from all datasets (data lake) which may contain portions of that profile as well as from Profile Store

[arijitg · Accepted solution]

Record Schema: The postman API does not support deletion of batches for datasets based
on record schemas. Due to the behavior of record type dataset, erroneous batches for
datasets based on record schemas cannot deleted and only way to correct those is to re-ingest the batch with the correct data to overwrite the incorrect records.

Time series: The postman API supports deletion of batches for datasets based on time-series
schemas. The batch ID must be included in the body of the POST request. This does not
delete or affect the data already residing in the Data Lake and hence there may be
inconsistencies between data lake and UPS.

If you need to delete the entire entity which represents all attributes and related time-series put together
from various sources to form a single representation on the Profile Store. The postman API
supports deletion of the entire entity from the Profile Store and may take up to 24 hours to
reflect. This method is recommendation for use cases where there is a requirement to retire /
delete inactive customers or identities. This does not delete or affect the data already residing in
the Data Lake and hence there may be inconsistencies between data lake and UPS.

Ref : Real-time Customer Profile API (adobe.com)

[sheejo]

@arijitg, @dwright , @Danny-Miller  , I have created an Privacy API call to delete an entry from profile( "jobId": "ad3bf439-a328-11ed-b1c3-d5bf6d987a8c" and "requestId": "16753630618421585RX-436") on 02/02/2023 06:37 PM GMT. Still it is showing the status as "Processing".   Above window says 24 hours.

Could you please let me know how many days this process will take?

[SivaK]

Hi,

Were you able to delete the profile with the privacy API call?  

[sheejo]

Yes, @maggod , you are correct. Need to delete 7000+ individual profiles from all datasets.

[dwright]

@sheejo ,

It sounds like you would want to use Data Hygiene for this. Assuming you have a list of primary ids, Data Hygiene allows you to delete the related data from data lake, profile, and identity service. The feature does require a Healthcare Shield or Privacy and Security Shield license: https://experienceleague.adobe.com/docs/platform-learn/tutorials/data-hygiene/delete-consumers.html?...

[sheejo]

Thanks @dwright 

[ChetanyaJain]

If you do not have Healthcare Shield, then raise Privacy requests with appropriate regulations for each profile.

You can do via UI [1] or via API [2]. This will basically create ticket for each profile, and backend team will delete it for you.

[1] https://privacyui.cloud.adobe.io/

[2] https://developer.adobe.com/experience-platform-apis/references/privacy-service/#tag/Privacy-jobs/op... 

Reference:

https://docs.adobe.com/content/help/en/experience-platform/privacy/home.html 

[sheejo]

Thanks @ChetanyaJain 

[sheejo]

For a deletion privacy request, how many days it will take to complete the process?

[ChetanyaJain]

Upto 30 days.

[Danny-Miller]

Please remember, Privacy Service is meant to help you manage customer data requests specifically.  Meaning it is meant to handle compliance with legal and organizational privacy regulations only.  Please use it for this only.

https://experienceleague.adobe.com/docs/experience-platform/privacy/regulations/overview.html?lang=e...

[ChetanyaJain]

@Danny-Miller, what is the option in the platform to delete profiles without paying an extra dime? Or can we raise a support ticket to delete the profiles?

[Nick_Hecht]

@ChetanyaJain  You would have to use the entity delete API. This will allow you to delete individual profiles in the profile store. You would have to que up the requests and process through them at 10 rps. Note this does not delete the profile data from the data lake, identity service, or downstream apps such as CJA its scope is the profile data store only. Privacy service deletes from all services but is for privacy requests only. Data hygiene deletes from all services but as you mention is a licensed service for running bulk operations. So as an alternate you can use entity delete API to delete single profiles from profile store. You can also use profile system delete jobs to delete full datasets or dataset batches that have been uploaded from profile store - deleting datasets and dataset batches from the data lake will also propagate to the profile store and identity store - but this approach is not ideal for when you need to delete specific IDs vs. entire datasets or dataset batches. 

Entity Delete API

https://developer.adobe.com/experience-platform-apis/references/profile/#tag/Entities/operation/retr...

Dataset Delete for profile enabled datasets

https://experienceleague.adobe.com/docs/experience-platform/catalog/datasets/user-guide.html?lang=en...

[navink386520]

@Nick_Hecht  @sheejo  @ChetanyaJain 

Folks the Entity Delete API is not part of Privacy API right. I am kinda lost as i have same requirements too.

Navin