What is the role of an identity field in an Experience Event schema? Can it be leverage for "upsert"?

[GabrielaNa1]

Hi, I am a beginner working with the Platform, and I am struggling a bit with the way data is ingested in datasets. Here are a few questions - any help will be really useful.

1. What is the role of the _id field? It seems to exist in any schema, but I don't see clear examples where actually data is mapped to it. I am not quite sure how the platform uses it and, because of that, I am not sure what kind of info should be populated in this field. I know that the field is mandatory for Experience Event schemas, but I fail to understand what it means. I created data sets and ingested multiple rows with the same value in _id, and there was no error. The rows were ingested every time, the only difference being in some of the metadata fields, but not in _id. So, not unique in the dataset.
2. I kind of understand the role of an Identity field when it comes to Record schemas, and them being enabled for profile. However, what is the role of an identity field for a Record schema not enabled for profile (like some lookup, for example), or in an Experience Event schema? I would like to elaborate on the later: for example, a schema for sales orders. When the order is created its status will probably be not shipped. Later one the status might change to shipped, and at another point in time becomes delivered. This are changes that happen to the same OrderID (my identity). Is there a way to keep one record in the dataset and just update the status, because so far, I see a new record is added every time, with the same OrderID (or _id, if I so choose to populate it with the value of the OrderID). In simple words, is there such notion of "upsert" with the datasets? Because so far all I can see is INSERT, which raises the question on how to get "the most recent set" for a particular identity value.

A bit of a different paradigm compared to relational Databases, and the documentation I found so far is not very clear in explaining this different behavior.

Sorry for the long message, but I wanted to provide as many details as possible, so maybe somebody who experienced the same challenges could related and help me find some answers.

Thanks.

[Rite18 · Accepted solution]

@GabrielaNa1  I hope below helps, considering Event Schema/Time series 

ID : serves as a unique identifier for each event hence supporting event tracking, uniqueness 

Timestamp: server the purpose to find the exact time when the event occurred during journey to support event sequencing, and profile stitching [time base merge profile as we]  

both would further help in identity resolution when identity fields are defined - where identities are passed to identity service to create an identity graph 

In case of  upsert or example would be good to have a object for shipping status and for each event set a value to this object rather doing upsert exiting value because , in time series event schema would be to store historical status 

[bjoern__koth]

Hi @GabrielaNa1 

as described here, it is system-generated once the event is received.

So, it has nothing to do with identity as such, if you are talking about identities in a profile context.