How does adobe detect duplicate events?

[takahiro_umezu]

Adobe can list certain event counted as duplicate. (duplicate_events)

Data column reference | Adobe Analytics

But how?
Checking with serialization number like event1:unique_id

or with event_list value (which contains othere events?)

or information in cookies?

We gave up event serialization because of the 20 bytes limit.
And instead of that we started to serialize using duplicate_events field.
So we would like to know how they can detect certain event as duplicate.
Thanks

[Jennifer_Dungan]

I assume you mean when you use the "Record once per Visit" on events:

I haven't seen how this presents in the Raw Data feeds, but I suspect that duplicate_events field in the Raw Data acts very much like event_list, passing a list of events that can be stripped from tracking due to duplication... 

If the data doesn't seem to line up with what you see in Workspace, you could attempt to calculate yourselves... basically, I think you would have to identify each visit (which would be a combination of visitor identification and visit number), then check for the first instance of the event in the visit, and ignoring all subsequent instances (but again, that should be already calculated by Adobe as part of the duplicate fields...)

[Jennifer_Dungan]

I believe all the duplication is calculated during processing. Based on the event configuration (whether "always track", "once per session" or "by event id") as data comes to the server, Adobe will store the list of events on each call, then if the event has a uniqueness logic applied, it will look at the session or at the serialization, determine if the event is duplicated, then add it to the duplication field... 

If the event is listed as a duplicate, the event will be ignored when pulling reports.

[takahiro_umezu]

Thanks for your reply.

I suppose like adobe has other table which stores events and can detect duplication rather than checking with event_list field (which contains other events in the very same record).