Experience Event Schema Best Practices

[RyanMoravick]

Hello! Im looking to see if anyone has some best practices around experience event schemas, specifically around a single event schema with multiple sources or datasets.

Use case: we have 3 paid media partners that we'd like to ingest engagement data back into AEP using source connectors/HTTP API? If each paid media source has 4 event types (conversions, impressions, clicks, video views), how would you leverage a single dataset for each paid media partners if fields such as timestamp needs to be unique to each event type source? Does there need to be a dataset for each event type per paid media source?

Thanks! 

[DavidRoss91]

Hi @RyanMoravick 

Multiple eventTypes in one dataset should not be a problem. For example we have ingested multiple Cart Events into one dataset. Each eventType shall still hold its own unique timestamp based on the event.

Hope this helps.

David

[RyanMoravick]

Hey @DavidRoss91 So you have a timestamp field for each eventType, how do you go about mapping to the mandatory timestamp field in the event schema? 

[_Manoj_Kumar_]

Hello @RyanMoravick  Either you can create your own timestamp field or use the existing one. It all depends on how you want to use this data and what is the business requirement.

If some event is happening on Paid media channels at X time, do you want to ingest the same timestamp in schema and then use it for segmentation Or timestamp is not used any where? 

If you want to use the timestamp coming from source then I would recommend to use the existing timestamp so that it represent the actual timing of the event happened on the source. Otherwise you can just use now() to ingest the timestamp of ingestion.

[RyanMoravick]

@DavidRoss91 I think im having trouble understanding how to map to the timestamp if there are multiple events present in the dataset. For example, ive attached a screenshot of click logEntryTime and conversion logEntryTime, which would map to the timestamp in the dataset, but I can't map more than one of these to the timestamp. 

[brekrut]

@RyanMoravick 

I can see you have a conversion timestamp and a click timestamp.  These are actually two different events from a user interaction. 

I would recommend breaking these out into two different records instead of pushing them into the same event.

A conversion and click may have different timestamps.

[RyanMoravick]

Thanks @brekrut by two different records, you mean two different datasets? So I would have a dataset for conversions, a dataset for clicks, etc?

[brekrut]

You could have this as one dataset but two different event types.  

Do you different TTL timeframes from a conversion event vs a click?

[brekrut · Accepted solution]

Hello @RyanMoravick 

As noted above when ingesting data into a experience event or time-series the following items are required:

a) a unique identifier of the event stored into the _id field.  Typically this is a guid, but this should be a unique value of the event.

b) timestamp - this would be timestamp of what has occurred in the past at the time of the event.  If you ingest data and use the timestamp ingestion time, then the segmentation logic will be used based upon this time and not the actual time of the event.  This can be an issue when there are data processing delays from systems.

c) Identify values - You should have a minimum of a single identity value to ensure the event is recored upon the profile store.  You also need to keep in mind how many events which are being considered for profile as there is a limit of 5K events per profile.

Another item to consider to is using the eventType for the event which you are ingesting. You can customize the eventTypes in the experience schema to classify the name of the event if it is not part of the XDM schema.