Adobe Experience Platform

sheejo · 2/1/23

Hello

We have a requirement to delete few profiles from AEP. What are the ways we can delete the profiles from AEP? If yes, how many profiles we can delete?

Please help.

Thanks in advance

Sheejo Rapheal

arijitg · 2/1/23

Record Schema: The postman API does not support deletion of batches for datasets based
on record schemas. Due to the behavior of record type dataset, erroneous batches for
datasets based on record schemas cannot deleted and only way to correct those is to re-ingest the batch with the correct data to overwrite the incorrect records.

Time series: The postman API supports deletion of batches for datasets based on time-series
schemas. The batch ID must be included in the body of the POST request. This does not
delete or affect the data already residing in the Data Lake and hence there may be
inconsistencies between data lake and UPS.

If you need to delete the entire entity which represents all attributes and related time-series put together
from various sources to form a single representation on the Profile Store. The postman API
supports deletion of the entire entity from the Profile Store and may take up to 24 hours to
reflect. This method is recommendation for use cases where there is a requirement to retire /
delete inactive customers or identities. This does not delete or affect the data already residing in
the Data Lake and hence there may be inconsistencies between data lake and UPS.

Ref : Real-time Customer Profile API (adobe.com)

View solution in original post

Nick_Hecht · 4/14/23

@navink386520 - Entity Delete API is a stand alone mechanism specific to the Profile Service that will delete profiles out of the profile service. It does not delete data out of the data lake, or identity store. However it is useful if for example you need to be sure profiles are removed so they are not marketed to. Perhaps the profile data was accidentally ingested or perhaps the profile is no longer a customer so you want to remove them - this is a good API to use for those use cases.

The Privacy API is a wrapper service that initiates delete requests in several downstream services including data lake, identity store, profile store, Customer Journey Analytics, Journey Optimizer, and other downstream applications. So it is one service you can leverage for removal across all services - but it is only to be leveraged for privacy related requests as it has specific performance guarantees associated with privacy related requests. In the background the privacy API will use the service specific mechanisms - such as the entity delete API for the profile service to delete data from the respective service. Think of the privacy API as a broker that initiates deletions across services. Whereas to delete data from a specific service you could use the service specific API - in this case the entity delete API.

Separately the data hygiene service acts similar to privacy API as a broker to remove data from one service to many downstream services - although its usage is focus on data hygiene and removal as opposed to privacy requests which the privacy service is focused on. https://experienceleague.adobe.com/docs/experience-platform/hygiene/home.html?lang=en

navink386520 · 4/20/23

Team,

I have critical scenario , thus am forced to explore all API that are discussed here and i have few ISSUES on API.

1. Batch Delete API

@arijitg : Firstly the batch delete API response i.e, based on Dataset ID , provides more that batch (not even associated with DatasetID) , Example: Dataset ID , lets say has 5 batches, API response says 50 batches w.r.t to Dataset ID.

If you say "associated batch" well my friend , there are relationship schema or association to that dataset.

So firstly "retrieve a list of batches" based on dataset id provides wrong api response.

2. Delete Entity API:

I have 4000 profiles to be deleted, there is no am going to use Delete Entity or Privacy API. I would rather delete it via Batch. However , as stated above its not working correctly.

Does any one else use Batch Delete API?

Nick_Hecht · 4/20/23

@navink386520 - It is important to distinguish that the "Batch Delete API" does not delete a batch of profiles. It deletes a "batch" in a dataset. A Batch being a autonomous unit of data that was ingested into a dataset - such as a file. A single file would be represented as a "batch" in a dataset. Batches in datasets are routinely consolidated hence why 50 may be ingested but only 5 shown.

When you describe your use case it seems you do not want to delete a "batch" from a dataset, it seems rather you want to delete a "batch" of profiles from the system - in other words a "bulk" of profiles. And it seems you want to do this via a file based method. Is that correct?

I am trying to understand why you are looking at the batch delete api when that is not the path to delete a selection of 4000 profiles that may or may not have been loaded into the system in one "batch".

navink386520 · 4/21/23

@Nick_Hecht thanks for response. As indicated , deleting 5000 profile using entity API will be decade long experience given, they are in multiple batches under same dataset.

Thus , idea is to use Delete Batch API. However, while 1st step in that is "Get list of Batch" using "Catalog Service API > Batches > Retrieve a list of batches" . The challenges is in 1st Step the is defect in Retrieve Batch list API (ticket reference - SALES0077584). The defect is what i mentioned in afore comments.

Nick_Hecht · 4/21/23

@navink386520 - the entity delete API deletes data from the profile store directly. The Delete Batch API deletes data from the data lake only - this will result in the profiles still being existent in the profile store.

We typically see customers write scripts using the entity delete API to process the API calls in bulk.

We typically see customers using the batch delete API to delete entire batches of data from the data lake - but not to delete specific profile records - as you point out that would be quite cumbersome.

If you need the data deleted from both the profile store and data lake I recommend you use entity delete API to delete from the profile store. and simply build a SQL query to create table with the specified records filtered out. Once the new table is created you can then delete the entire batches or datasets with the data that you would like removed.

We are working on a data hygiene feature that would help do this across services in bulk. If you would like to learn more about this please reach out to your account team to have time set up with our product management team in this area.