Delete profiles from AEP | Community
Skip to main content
sheejo
sheejo
Level 3
February 1, 2023
Solved

Delete profiles from AEP

  • February 1, 2023
  • 6 replies
  • 15567 views

Hello

We have a requirement to delete few profiles from AEP. What are the ways we can delete the profiles from AEP? If yes, how many profiles we can delete?

Please help.

 

Thanks in advance

Sheejo Rapheal

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by arijitg

    I don't think batch deletion API will solve this request. Deleting a batch doesn't mean deleting the profile.

    I understand @sheejo wants to delete 7000 individual profiles (not batches, profiles are known by their ID) from all datasets (data lake) which may contain portions of that profile as well as from Profile Store


    Record Schema: The postman API does not support deletion of batches for datasets based
    on record schemas. Due to the behavior of record type dataset, erroneous batches for
    datasets based on record schemas cannot deleted and only way to correct those is to re-ingest the batch with the correct data to overwrite the incorrect records.


    Time series: The postman API supports deletion of batches for datasets based on time-series
    schemas. The batch ID must be included in the body of the POST request. This does not
    delete or affect the data already residing in the Data Lake and hence there may be
    inconsistencies between data lake and UPS.

     

    If you need to delete the entire entity which represents all attributes and related time-series put together
    from various sources to form a single representation on the Profile Store. The postman API
    supports deletion of the entire entity from the Profile Store and may take up to 24 hours to
    reflect. This method is recommendation for use cases where there is a requirement to retire /
    delete inactive customers or identities. This does not delete or affect the data already residing in
    the Data Lake and hence there may be inconsistencies between data lake and UPS.

    Ref : Real-time Customer Profile API (adobe.com)

    6 replies

    arijitg
    Adobe Employee
    arijitgAdobe Employee
    Adobe Employee
    February 1, 2023

    Depending on your requirement you can delete them by entire dataset or by a batch or even by individual profiles based on namespace.

    sheejo
    sheejoAuthor
    Level 3
    February 1, 2023

    Thanks for your reply. How can we do the deletion? Is there any API for that or any other process?

    arijitg
    Adobe Employee
    arijitgAdobe Employee
    Adobe Employee
    February 1, 2023

    I can answer better if you can tell me how do you want to delete them.

    By Dataset/Batch/Namespace?

    D
    Adobe Employee
    dwright-adobeAdobe Employee
    Adobe Employee
    February 1, 2023

    @sheejo ,

    It sounds like you would want to use Data Hygiene for this. Assuming you have a list of primary ids, Data Hygiene allows you to delete the related data from data lake, profile, and identity service. The feature does require a Healthcare Shield or Privacy and Security Shield license: https://experienceleague.adobe.com/docs/platform-learn/tutorials/data-hygiene/delete-consumers.html?lang=en

    sheejo
    sheejoAuthor
    Level 3
    February 1, 2023

    Thanks @dwright-adobe 

    ChetanyaJain-1
    Community Advisor
    ChetanyaJain-1Community Advisor
    Community Advisor
    February 2, 2023

    If you do not have Healthcare Shield, then raise Privacy requests with appropriate regulations for each profile.

    You can do via UI [1] or via API [2]. This will basically create ticket for each profile, and backend team will delete it for you.

     

    [1] https://privacyui.cloud.adobe.io/

    [2] https://developer.adobe.com/experience-platform-apis/references/privacy-service/#tag/Privacy-jobs/operation/createPrivacyJob 

     

    Reference:

    https://docs.adobe.com/content/help/en/experience-platform/privacy/home.html 

    sheejo
    sheejoAuthor
    Level 3
    February 3, 2023

    Thanks @chetanyajain-1 

      N
      navin_kumar0601
      Level 4
      April 20, 2023

      Team,

      I have critical scenario , thus am forced to explore all API that are discussed here and i have few ISSUES on API.

      1. Batch Delete API

      @arijitg : Firstly the batch delete API response i.e, based on Dataset ID , provides more that batch (not even associated with DatasetID) , Example: Dataset ID , lets say has 5 batches, API response says 50 batches w.r.t to Dataset ID. 

      If you say "associated batch" well my friend , there are relationship schema or association to that dataset. 

      So firstly "retrieve a list of batches" based on dataset id provides wrong api response.

      2. Delete Entity API:

      I have 4000 profiles to be deleted, there is no am going to use Delete Entity or Privacy API. I would rather delete it via Batch. However , as stated above its not working correctly.

       

      Does any one else use Batch Delete API?

       

       

      N
      Adobe Employee
      Nick_HechtAdobe Employee
      Adobe Employee
      April 20, 2023

      @navin_kumar0601  - It is important to distinguish that the "Batch Delete API" does not delete a batch of profiles. It deletes a "batch" in a dataset. A Batch being a autonomous unit of data that was ingested into a dataset - such as a file. A single file would be represented as a "batch" in a dataset. Batches in datasets are routinely consolidated hence why 50 may be ingested but only 5 shown.

      When you describe your use case it seems you do not want to delete a "batch" from a dataset, it seems rather you want to delete a "batch" of profiles from the system - in other words a "bulk" of profiles. And it seems you want to do this via a file based method. Is that correct?

      I am trying to understand why you are looking at the batch delete api when that is not the path to delete a selection of 4000 profiles that may or may not have been loaded into the system in one "batch".

      N
      navin_kumar0601
      Level 4
      April 21, 2023

      @nick_hecht  thanks for response. As indicated , deleting 5000 profile using entity API will be decade long experience given, they are in multiple batches under same dataset.

      Thus , idea is to use Delete Batch API. However, while 1st step in that is "Get list of Batch" using "Catalog Service API > Batches > Retrieve a list of batches" . The challenges is in 1st Step the is defect in Retrieve Batch list API (ticket reference - SALES0077584).  The defect is what i mentioned in afore comments. 

      N
      Adobe Employee
      Nick_HechtAdobe Employee
      Adobe Employee
      April 21, 2023

      @navin_kumar0601 - the entity delete API deletes data from the profile store directly. The Delete Batch API deletes data from the data lake only - this will result in the profiles still being existent in the profile store. 

      We typically see customers write scripts using the entity delete API to process the API calls in bulk. 

      We typically see customers using the batch delete API to delete entire batches of data from the data lake - but not to delete specific profile records - as you point out that would be quite cumbersome. 

      If you need the data deleted from both the profile store and data lake I recommend you use entity delete API to delete from the profile store. and simply build a SQL query to create table with the specified records filtered out. Once the new table is created you can then delete the entire batches or datasets with the data that you would like removed. 

      We are working on a data hygiene feature that would help do this across services in bulk. If you would like to learn more about this please reach out to your account team to have time set up with our product management team in this area. 

       

      N
      nirshani
      Level 3
      December 20, 2023

      hi 

      please see this new feature (Beta  for now ) 

      https://experienceleague.adobe.com/docs/experience-platform/data-lifecycle/ui/record-delete.html?lang=en#

      i think it is what you need

      Terms & ConditionsAccessibility statement

      Loading footer...