Handling events in profiles and datalake

Hi @buthpurkiran ,

Specific scenarios haven’t come up for me yet, but one option is to use query service to isolate the events you want, write them into a new dataset, and then enable that dataset for Real-Time Profile ingestion.

Specifically, I found the above approach in the first paragraph of the reference link below.

https://experienceleague.adobe.com/en/docs/experience-platform/query/home

Thanks,

Ankit

Hi @buthpurkiran ,

Your idea to retain or “rehydrate” specific event data beyond the profile TTL (Time-To-Live) using SQL in the Data Lake is absolutely valid and aligns with how many advanced Adobe RTCDP users structure their data lifecycle.

Common Use Cases

1. Re-evaluation of Profile Behavior Over Longer Windows

   • E.g., "Users who viewed product A 3 months ago and now came via email"

   • Event expired from profile, but still present in Data Lake

2. Backfill Audience Segments

   • Historical re-segmentation based on new logic or campaign timing changes

3. Custom Aggregations for Summary Datasets

   • Summarize specific events per user per month and store that back into a profile-enabled dataset (summarized)

4. Data Science/Attribution Use Cases

   • Feature engineering on long-term behavioral signals not available in profile store

Best Practices

1. Create a Profile-enabled summary dataset

   • Schema: XDM ExperienceEvent or Custom Schema

   • Include: Identity, event type, and minimal attributes

2. Schedule the query in Query Service

   • Run daily or weekly to process "expired" but relevant event data

3. Tag these rehydrated events clearly

   • Use a marker field (e.g., event_source = 'rehydrated_from_datalake') to differentiate

4. Avoid overloading Profile with too much legacy data

   • Use summarized or roll-up records (e.g., “last purchased in past 90 days”)

@buthpurkiran Just checking in — were you able to resolve your issue?
We’d love to hear how things worked out. If the suggestions above helped, marking a response as correct can guide others with similar questions. And if you found another solution, feel free to share it — your insights could really benefit the community. Thanks again for being part of the conversation!