The source of the profile_export_for_destination_merge_policy__XXX is all record based datasets which have been marked for profile.

Please refer to the following digram which outlines the latency through each component.https://experienceleague.adobe.com/en/docs/blueprints-learn/architecture/architecture-overview/deployment/guardrails#end-to-end-latency-diagrams

Hello @Indra  Are you requesting the processing time of data which was ingested into datalake and then promoted upon the profile store?

Hello @Indra  The profile_export_for_destination_merge_policy is updated on periodic basis as profile attributes are updated. The Profile_Snapshot is updated 1 x / day after the segmentation evaluation job completes.

The managing of segments is controlled via profile management.  This will control the viewing and creating of segment. The activation of segments is controlled within the destinations section of role based access control.

Hello @SaurabhCh3  if you have data in dataSet A under the path of _abc.phone.number.number I would create an a query similar to the following.  Each object should be represented by a struct. insert into dataSetB select struct(              struct(              struct(          (x.number as number) ...

@SahuSa1 Can I ask why the source is producing duplicate rows of data with the latest timestamp? If the data is not changing why is the data being re-ingested?

The merge policy has an impact on the profile store.  What is the use case you are attempting to achieve with audiences of two different merge policies.

The record based profile will write data upon the dataset and the last write of the data will be available on profile.  I would consider an audit timestamp with the xdm profile schema to track when changes are applied on the dataset instead of the batch_id

@GabrielaNa1  What is your use case to store the batchid upon profile?