Data source for Segmentation, Unique Profiles and Deduplication

[mdajaz]

Per my understanding, Segmentation works on Unified Profile datalake and NOT on the main AEP datalake where we have all the historical records.

In that case, all the de-duplication should have already been taken care by Identity service when it is doing all the merging of different/duplicate profile records assuming that the Identities have been set appropriately.

Is that fair to assume, that scenarios where identities have not been set properly and Profile Service cannot merge different records, implies multiple records for the same customer still exists post Profile unification. In this case, when we want to send unique records to Destination (based on attribute(s)), De-Duplication feature is introduced for this use-case.

Please confirm.

[abhinavbalooni · Accepted solution]

Hey @mdajaz 

That's a very interesting observation. It does look like what you've mentioned is correct. Let's hear from others but definitely a valid point !

Cheers,

Abhinav

[sreeCharan81]

Seems a reasonable assumption. However I am thinking, since this is relevant only in file based destinations (Profile Export), AEP might explode the identity Map(which is a complex Object[]), so it would generate duplicate records for the same profile and hence the deduplication logic. Just thinking out loud here.

[NimashaJain]

@mdajaz Did you find solution? It would be great if share it with wider audience in the community.