Need clarifications on Profile Snapshort Export

[ShashiCh]

We see in the dev environment the profile snapshot export job is scheduled to run at 4:30AM UTC Everyday (12:30AM EST) Below is the configured expression.

"name": "Profile-Snapshot-Export-Schedule-aaa-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",

    "state": "inactive",

    "type": "export",

    "schedule": "0 30 4 * * ?",

When we observe the dataset 'Profile-Snapshot-Export-zzzzz-zzzzzzzz-zzzzzzz', the ingestion gets successful at 11AM EST (3AM UTC), that's when we see data in the query service (data lake). There are 37 million profiles are there in the system but on a specific day it shows 700k profiles being updated through profile snapshot.

1. What is ingested timestamp in the dataset screenshot?

2. Why it takes almost 10.5 hours to process. Because the configured time 4:30AM UTC (12:30AM EST) and profile data reflection happens at 3AMUTC (11AM EST)

3. In the dataset UI, we observe the dataset shows loading status only after 10:30AM EST?

4. How much time it takes an average for profile snapshot export job to be completed?

[DavidRoss91 · Accepted solution]

Hello @ShashiCh 

The profile snapshot export job kicks off immediately following your global segment evaluation. For example, GSE finishes around 3am everyday and the Profile Snapshot job would trigger within seconds of that completing (see attachment)

1. the ingested timestamp matches the time the snapshot export job kicks off

2./3. I believe these will both depend on the timing of your Global Segmentation Evaluation job and the completion time

4. we usually see about 1.5 hrs for the job to complete.

[ShashiCh]

@DavidRoss91 - This makes sense. Thanks for the response.

From the above clarification it seems that the below schedule 4:30AM UTC does not have any contribution here because profile snapshot export always happens after the completion of GSE and GSE is totally dependent on the number of audiences.

OR

if we change the status to active, will it run on its own schedule? If yes, Are there any impacts if snapshot export schedule time is before GSE start time? OR Can we schedule to run snapshot export job and GSE Start time run parallelly?

"name": "Profile-Snapshot-Export-Schedule-aaa-zzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",

    "state": "inactive",

    "type": "export",

    "schedule": "0 30 4 * * ?",