Skip to main content
ShashiCh
Level 3
June 22, 2024
Solved

Need clarifications on Profile Snapshort Export

We see in the dev environment the profile snapshot export job is scheduled to run at 4:30AM UTC Everyday (12:30AM EST) Below is the configured expression.

 

"name": "Profile-Snapshot-Export-Schedule-aaa-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",
    "state": "inactive",
    "type": "export",
    "schedule": "0 30 4 * * ?",
 
When we observe the dataset 'Profile-Snapshot-Export-zzzzz-zzzzzzzz-zzzzzzz', the ingestion gets successful at 11AM EST (3AM UTC), that's when we see data in the query service (data lake). There are 37 million profiles are there in the system but on a specific day it shows 700k profiles being updated through profile snapshot.

 

1. What is ingested timestamp in the dataset screenshot?
2. Why it takes almost 10.5 hours to process. Because the configured time 4:30AM UTC (12:30AM EST) and profile data reflection happens at 3AMUTC (11AM EST)
3. In the dataset UI, we observe the dataset shows loading status only after 10:30AM EST?
4. How much time it takes an average for profile snapshot export job to be completed?
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
DavidRoss91
DavidRoss91Accepted solution
Community Advisor
June 24, 2024

Hello @shashich 

The profile snapshot export job kicks off immediately following your global segment evaluation. For example, GSE finishes around 3am everyday and the Profile Snapshot job would trigger within seconds of that completing (see attachment)

 

1. the ingested timestamp matches the time the snapshot export job kicks off

2./3. I believe these will both depend on the timing of your Global Segmentation Evaluation job and the completion time

4. we usually see about 1.5 hrs for the job to complete.

 

 

David Ross
View in thread

2 replies

DavidRoss91
Community Advisor
DavidRoss91Community AdvisorAccepted solution
Community Advisor
June 24, 2024

Hello @shashich 

The profile snapshot export job kicks off immediately following your global segment evaluation. For example, GSE finishes around 3am everyday and the Profile Snapshot job would trigger within seconds of that completing (see attachment)

 

1. the ingested timestamp matches the time the snapshot export job kicks off

2./3. I believe these will both depend on the timing of your Global Segmentation Evaluation job and the completion time

4. we usually see about 1.5 hrs for the job to complete.

 

 

Translate
David Ross
    ShashiChAuthor
    Level 3
    June 24, 2024

    @davidross91 - This makes sense. Thanks for the response.

     

    From the above clarification it seems that the below schedule 4:30AM UTC does not have any contribution here because profile snapshot export always happens after the completion of GSE and GSE is totally dependent on the number of audiences.

    OR

    if we change the status to active, will it run on its own schedule? If yes, Are there any impacts if snapshot export schedule time is before GSE start time? OR Can we schedule to run snapshot export job and GSE Start time run parallelly?

     

    "name": "Profile-Snapshot-Export-Schedule-aaa-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",
        "state": "inactive",
        "type": "export",
        "schedule": "0 30 4 * * ?",
    Translate
    Terms & ConditionsAccessibility statement

    Loading footer...