Expand my Community achievements bar.

SOLVED

Issue in identifying latest information of a type of Experience Event

Avatar

Level 2

Hello Team,

 

Need some guidance on resolving one issue related to ExperienceEvent while using them in creating Segment. Though it is mutable, we had to choose one of my dimension data as ExperienceEvent as that is having 1-to-many relationship with my customer record (array will also not work as number of records are sometimes more than 100 per customer record). For example: customer has many accounts, and account details (example: status, monthly average balance) can get changed in time. So one customer can have multiple events for a same account with different values in a certain timeframe. In this situation, how to get the latest event of the account, if I have to build a segment based on the latest information of the account?

 

Also looking for some suggestion if there is any better way of modeling the XDM schemas for 1 to many relationships for mutable dimensions ? Array option as Record schema wont work, as during update via real time or batch, we have to rebuild the whole array.

 

Thanks in advance for the help.

1 Accepted Solution

Avatar

Correct answer by
Employee

@bitunsen-2022 its a bit hard to grok the issue without a diagram but I'll do my best to answer this.

 

In the relational world what you are describing a 1:N relationship to another entity object called account. It would typically be modeled like you see below.  With Experience Platform we only support a single entity defined as Person which is composed of fragments of either traits of that person (XDM Individual Profile) or behaviors of that person (XDM Experience Event). In the relational world to bridge the gap between two different entities you would build a bridge table as you see below with Customer Account.  

ericknee_0-1647009315739.png

 

When you go to model this in Experience Platform you cannot think relationally anymore as our system is built on NoSQL. This means that other entities have to be denormalized back into one of the fragment types (XDM Individual Profile or XDM Experience Event).  Below is what we would be expected to be modeled:

ericknee_1-1647009748807.png

 

You have to denormalize the bridge table and the entity into an array on the profile if you want to know the current accounts each person owns. This does mean you have to think through the data engineering side of this.

  • Batch feeds - you will need to perform some form of data engineering to also ensure you re-create this array each time you update the customer record.
    • (optionally) - you could create two datasets (XDM Individual Profile) where one dataset contains the customer traits minus the account details and another dataset contains the cusId + account details in an array.
  • Stream feeds - you will need to build a serverless function to construct the array of accounts and append to the customer record any time you have an update if you are doing full record re-instatement
    • (optionally) - create two datasets like in batch but just stream the pieces in separately 

 

The only reason to model accounts as events in this scenario is if there is a business case where the customer is asking the history of the how account is changing over time. That doesn't appear to be the case here so I would not model this as an event as that is the intention from the relational model.

 

View solution in original post

4 Replies

Avatar

Correct answer by
Employee

@bitunsen-2022 its a bit hard to grok the issue without a diagram but I'll do my best to answer this.

 

In the relational world what you are describing a 1:N relationship to another entity object called account. It would typically be modeled like you see below.  With Experience Platform we only support a single entity defined as Person which is composed of fragments of either traits of that person (XDM Individual Profile) or behaviors of that person (XDM Experience Event). In the relational world to bridge the gap between two different entities you would build a bridge table as you see below with Customer Account.  

ericknee_0-1647009315739.png

 

When you go to model this in Experience Platform you cannot think relationally anymore as our system is built on NoSQL. This means that other entities have to be denormalized back into one of the fragment types (XDM Individual Profile or XDM Experience Event).  Below is what we would be expected to be modeled:

ericknee_1-1647009748807.png

 

You have to denormalize the bridge table and the entity into an array on the profile if you want to know the current accounts each person owns. This does mean you have to think through the data engineering side of this.

  • Batch feeds - you will need to perform some form of data engineering to also ensure you re-create this array each time you update the customer record.
    • (optionally) - you could create two datasets (XDM Individual Profile) where one dataset contains the customer traits minus the account details and another dataset contains the cusId + account details in an array.
  • Stream feeds - you will need to build a serverless function to construct the array of accounts and append to the customer record any time you have an update if you are doing full record re-instatement
    • (optionally) - create two datasets like in batch but just stream the pieces in separately 

 

The only reason to model accounts as events in this scenario is if there is a business case where the customer is asking the history of the how account is changing over time. That doesn't appear to be the case here so I would not model this as an event as that is the intention from the relational model.

 

Avatar

Level 2

Thanks a lot Erick for your detailed level of response. I really appreciate the way you responded. You appropriately represented the model. 

 

I was bit hesitant in representing Accounts using array in profile, because I have some customers who are having more than 5000 accounts. I know those are edge cases (3% of total customer population), still wanted to understand if there is any size limit or restrictions per profile for an array (I am aware of the total size limit per profile following the guardrails).

 

 

Avatar

Level 2

How did you resolve this and what is the approach you took to address this issue?
I have the similar situation to get the latest status of the subscription and segment on top of it.
But I also know that array's have some storage limitations. 

Avatar

Employee

@bitunsen-2022 there is a limitation the array size which you read about in the docs here. In short arrays should be <500 elements deep if you are de-normalizing back into a XDM Individual Profile class-based schema.  I question the value of modeling accounts as array here because unless you have a business use case you are just creating bloat for an engineering exercise.  Usually what I've seen is array's created that hold the most recent 10, last 10, etc.  

 

What is the business use case for bringing the accounts into the model?  Are you modeling for a B2B scenario?  

 

Additional context is needed to provide direction beyond what I've shared above as the business case is incredibly important here.