Issue with Record Deletion – Not Cleaning Up from Datalake | Community
Skip to main content
U
user48
Level 2
January 1, 2025
Solved

Issue with Record Deletion – Not Cleaning Up from Datalake

  • January 1, 2025
  • 4 replies
  • 1110 views

Hi Community,

I’m encountering an issue while trying to delete records in our sandbox and would appreciate any insights or help.

Here’s the situation:

  • I created a delete record request via the UI for two records in our sandbox. The request was completed successfully, but the records were only cleaned up from UPS and not from the datalake.
  • Additionally, according to the documentation, I am unable to find the option to select whether to delete records from a specific dataset or from all datasets.

Has anyone faced a similar issue or knows how to resolve this? Any guidance would be greatly appreciated.

I’ve attached screenshots for reference:

  1. Screenshot of the successful completion of the delete request
  2. Screenshot from UI showing the absence of the dataset selection functionality as per document.
  3. Screenshot of the dataset selection option from the AEP document

Thanks in advance!

Best answer by user48

Sure @kautuk_sahni.

Thank you, @harveer_singhgi1  and @davidross91 , for your help. I reached out to Adobe, and they clarified that for records to be deleted from the Data Lake, the organization needs to be 'Delta-migrated.' However, even for Delta-migrated organizations, hygiene requests are not processed against the Data Lake, as this capability is still in 'limited release' and not yet generally available.

Currently, to enable a full-service level for the Data Lake, we need to raise a ticket and provide the necessary confirmation.

 

4 replies

Harveer_SinghGi1
Community Advisor
Harveer_SinghGi1Community Advisor
Community Advisor
January 2, 2025

Hi @user48 ,

The option to choose single or multiple datasets is available to organizations who have been moved to the Delta format. If you are not seeing this option then most probably your organization have not undergone delta migration yet. I suspect that record deletion from only UPS and not data lake might also be linked to the same reason but not 100% sure. As record deletion is a beta feature we don't have a lot of documentation around it's workings. I'd suggest you reach out to support team for more clarity on first part of your query.

Cheers!

    DavidRoss91
    Community Advisor
    DavidRoss91Community Advisor
    Community Advisor
    January 2, 2025

    To tag onto @harveer_singhgi1 's answer--

    The doc states:

    To improve efficiency and make dataset operations less expensive, organizations who have been moved to the Delta format can delete data from the Identity Service, Real-Time Customer Profile, and the data lake. This type of user is referred to as delta-migrated. Users from organizations who have been delta-migrated can choose to delete records from either a single or all datasets. Users from organizations that have not undergone delta migration are unable to selectively delete records from either a single dataset or all datasets

     

    So, it would appear that this is the reason for your issues you are having.

      U
      user48Author
      Level 2
      January 2, 2025

      Thank you @davidross91 and @harveer_singhgi1 ,

      Could you please help clarify what "Delta Migrated" refers to? Additionally, how do I find out if our organization has been migrated to the Delta format?

       

      If we have not been migrated, could you please explain the process for migrating to Delta?

       

      DavidRoss91
      Community Advisor
      DavidRoss91Community Advisor
      Community Advisor
      January 2, 2025

      Hi @user48 

      I pulled this basic definition from MS documentation:

      "Delta Format is an open-source storage layer that brings reliability to data lakes"

       

      I would recommend reaching out to Adobe Support through a ticket, to gather more information direct to your organization for further details on the delta migration.

       

        kautuk_sahni
        Community Manager
        kautuk_sahniCommunity Manager
        Community Manager
        January 20, 2025

        @user48 Did you find the suggestions helpful? Please let us know if you require more information. Otherwise, please mark the answer as correct for posterity. If you've discovered a solution yourself, we would appreciate it if you could share it with the community. Thank you!

        Kautuk Sahni
        U
        user48AuthorAccepted solution
        Level 2
        January 21, 2025

        Sure @kautuk_sahni.

        Thank you, @harveer_singhgi1  and @davidross91 , for your help. I reached out to Adobe, and they clarified that for records to be deleted from the Data Lake, the organization needs to be 'Delta-migrated.' However, even for Delta-migrated organizations, hygiene requests are not processed against the Data Lake, as this capability is still in 'limited release' and not yet generally available.

        Currently, to enable a full-service level for the Data Lake, we need to raise a ticket and provide the necessary confirmation.

         

          Terms & ConditionsAccessibility statement

          Loading footer...