Does TTL affect customer AI? | Community
Skip to main content
SahuSa1
SahuSa1
Level 3
November 18, 2024
Solved

Does TTL affect customer AI?

  • November 18, 2024
  • 1 reply
  • 1100 views

From the AEP Architecture diagram I think Customer AI works at datalake level but not at profile store level, so I think TTL set on an events type dataset will not affect customer AI data requirements. Please let me know if my understanding is correct. 

 

Thanks

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Parvesh_Parmar

Hello @sahusa1 , 

 

I have just cross-checked Adobe's documentation on Experience Event Expirations (Experience Event Expirations | Adobe Experience Platform).

According to the documentation, Experience Event expirations will delete data from the Profile Store, but not from the Data Lake.

Regarding Pseudonymous Profiles, while it is not explicitly mentioned, I believe it follows the same functionality. Here’s the relevant documentation for your reference: Pseudonymous Profile data expiration | Adobe Experience Platform.

If your customer is directly using Data Lake data for training data models or predictions, or if any out-of-the-box Adobe data model is running on the Data Lake, there should be no impact. However, if you are planning to build something that relies on event data and pseudonymous data and you have expiration rules set, I recommend cross-checking this with Adobe for confirmation. That way, if any issues arise, they can provide the necessary support.

Kind regards,
Parvesh

 

1 reply

Parvesh_Parmar
Community Advisor
Parvesh_ParmarCommunity Advisor
Community Advisor
November 18, 2024

Hello @sahusa1 ,

Your understanding is mostly correct, but let me provide additional clarity:

Customer AI and Data Lake Operations:

Customer AI relies on data stored in the data lake for training its models and making predictions. If datasets in the data lake are managed with expiration policies (TTL), the availability of critical data for Customer AI can be impacted. However, if expiration dates or processing rules are configured properly, data will not be deleted prematurely.

Best Practices for Managing Data Retention for Customer AI:

  1. Align Data Retention with Customer AI Requirements:

    • If you anticipate that Customer AI models require one year of historical data, ensure the data is retained in the data lake for at least one year.
    • You can achieve this by extending TTL settings or disabling TTL for datasets critical to Customer AI operations.

  2. Understand the Purpose of AEP:

    • AEP is a data platform designed for customer experience management, stitching the data from different source, segmentation and activation,  not a long-term data warehouse.
    • If your use case requires long-term data storage for Customer AI or other tools, it is best to manage historical data outside of AEP in a dedicated data warehouse or data lake solution.
    • If you keep data for long term in DWH or other cloud like azure or AWS and run these models there for customer AI, you will have more performance, customization power, budget friendly as compare to do in AEP.

 

Hope it will help!

 

Best regards,

Parvesh

Parvesh Parmar – Adobe Community Advisor https://www.linkedin.com/in/parvesh-parmar/
    SahuSa1
    SahuSa1Author
    Level 3
    November 19, 2024

    Hi @parvesh_parmar , Thank you so much for your quick response.

     

    Please correct me if I am wrong, I had the understanding that TTL only impacts the profile store and it does not affect the data stored in the datalake. Only I guess, the pseudonymous profile data would get expired from both profile store and datalake if it has no activity within the TTL set period, due to its "all or nothing" concept.

     

    (Source - https://www.youtube.com/watch?v=We3RN-41Dik)

     

    So,TTL set on dataset should not affect the data requirements for customer AI right? 

    Also, can TTL be applied so that data also gets expired from datalake along with profile store? 

     

    Please enlighten me.

     

    Thanks,

    Sambit Sahu

      Parvesh_Parmar
      Community Advisor
      Parvesh_ParmarCommunity AdvisorAccepted solution
      Community Advisor
      November 22, 2024

      Hello @sahusa1 , 

       

      I have just cross-checked Adobe's documentation on Experience Event Expirations (Experience Event Expirations | Adobe Experience Platform).

      According to the documentation, Experience Event expirations will delete data from the Profile Store, but not from the Data Lake.

      Regarding Pseudonymous Profiles, while it is not explicitly mentioned, I believe it follows the same functionality. Here’s the relevant documentation for your reference: Pseudonymous Profile data expiration | Adobe Experience Platform.

      If your customer is directly using Data Lake data for training data models or predictions, or if any out-of-the-box Adobe data model is running on the Data Lake, there should be no impact. However, if you are planning to build something that relies on event data and pseudonymous data and you have expiration rules set, I recommend cross-checking this with Adobe for confirmation. That way, if any issues arise, they can provide the necessary support.

      Kind regards,
      Parvesh

       

      Parvesh Parmar – Adobe Community Advisor https://www.linkedin.com/in/parvesh-parmar/
        Terms & ConditionsAccessibility statement

        Loading footer...