XDM Identity field as array

[BitunSenAEP]

I have a data scenario, where my primary Identity has multiple secondary identities in the source data.

There is a MDM system which is generating the Primary Identity - Customer Id

The subscription system is a standalone system which is managing various subscriptions and also generate an identitifer for each customer "Subscriber Id". Now the problem is, because of not havign strong matching engine and varity of data, alsmost for 20% of cases, for same person, this system ends up creating multiple "Subscriber Id" for a person.

Illustration:

MDM

Thomas Cook and Tom Cook are identified as same customer and ends up having one Customer Id (12345679)

Subscription System

Due to the different first name, this platform generated 2 separate Subscriber Ids 

• Thomas Cook : Subscriber Id: 50001
• Tom Cook : Subscriber Id: 60001

Subscription system also sends various events to AEP and shares the Subscriber Id (Primary Identity) as event data (subscrition related call data etc)

MDM System mainatains a cross reference table to maintain the cross referrence between Customer Id and Subscriber Id, as follows:

Customer Id       Subscriber Id

12345679.          50001

12345679.          60001

Some times, Subscription system deletes the Subscriber Id as well.

In AEP, we need to consume this cross referrence table so that we can stitch the Customer Data from MDM with the Subscription data. Now the question is: shall we create

• OPTION 1: an Event Schema with Customer Id as Primary Identity and Subscriber Id (as non-primary identity)
  • With Event schema we can maintain 1:N relationship between Customer Id andSubscriber Id
  • Problem: if Subscriber Id gets deleted from Subscription system, then that need to be deleted from AEP as well, then we need to raise Data Lifecycle request to delete all the records having the deleted Subscriber Id. It takes around 7-15 days to complete. Not a nail biting issue as it will not increase the number of profiles in AEP
• OPTION 2: a Individual Profile schema with Customer Id as Primary Identity and an array of Subscriber Id (as non-primary identity)
  • With this option, we can also maintain 1:N relationship between Customer Id andSubscriber Id
  • Problem: Any time Subscriber Id is getting deleted from Subscription system, then we need to request MDM system to rebuild the array of Subscriber Id for the Customer Id associated with the deleted Subscriber Id. Very low impact issue.
• OPTION 3: a Individual Profile schema with identityMap where Customer Id as Primary Identity and an array of Subscriber Id (as non-primary identity)
  • PROS and CONS are same as Option 2

Trying to understand which option will be the best to implement, considering we will be using these data in:

• Customer Journey Analytics
• Adone Journey Optimizer
• RT-CDP - Segmentation

Looking forward to see some valuable suggestions from the experts here in this community.

[BitunSenAEP]

One additional information: Subscription system is very old and cannot have the MDM generated Customer Id in their system.

[TylerKrause]

Option 2 Spinoff: Can you instead just ingest separate records for each example of the subscriber ID and just have a 1:N Relationship that gets stored in UPS. This would allow you to additionally more cleanly have a use for Subscriber ID as a Secondary, Non-Primary key, similar to how ECID works in many envrionments.

[BitunSenAEP]

Thanks a lot @TylerKrause - not a bad idea. This can be an option as well. 

Any ways, in all the options, for the delete scenario, we have to go with Data Lifecycle request all the time.

[TylerKrause]

Do you get a clean programatic feed of those unsubcriptions? If so, you could scope out automation of the deletes utilizing the Data Hygine API. 

Right now, it's use for individual record deletes is locked behind a beta, but you could either scope out trying to get into it via support from product, OR you could schedule weekly/monthly automation for clearing out expired records, and simply log the expired records in either a separate dataset, or with an expired attribute that you can utilize to exclud via your audiences, reports, journeys, etc.

Data Hygiene API Guide | Adobe Experience Platform

[BitunSenAEP]

Yes @TylerKrause - I have written a python script which does the following:

• Read a CSV file which contains list of Identity and respective Identity Namespace
• split them into 100K buckets
• for each bucket
  • Create the Data Hygene API request and invoke the Data Hygene API

[TylerKrause]

Sounds good! I hope that can help you keep it organized. Let me know if you have any other thoughts you would like a perspective on!

Tyler

[BitunSenAEP · Accepted solution]

Hi @TylerKrause - Just to share another realization with the community - Option 2 and Option 3 should not be considered as an option, when Identity values can change or need to be deleted using Data Lifecycle requests. The reason being, Data Lifecycle request deleted the record which is having the requested Identity. For the array based Identity, it will not be possible to delete one single item from the array.

So, I am going with Option 1 (Event based). As I had mentioned earlier, your solution will also work fine. Its just that, the data in the profile store would look bit confusing, unless we notice the content of IdentityMap.

Thanks again for your time in helping me out and pushing me thinking all aspects.