we are currently developing a component that moves the data from Adobe campaign database to somewhere else in nearly realtime for dataware housing purpose.
We cannot use CDC technology such as Debezium for some technical constraints, so this leaves us only one option that polls the adobe database directly. we need to capture new record as well as the update on a particular delivery(such as status change: fail, success, pending etc.)
One solution is to poll "TSLASTMODIFIED" timestamp, and keep track of the max "IBROADLOGID".
If one record has IBROADLOGID > last MAX (IBROADLOGID) then it is a new record, then do some logic
if one record has IBROADLOGID <= last MAX(IBROADLOGID),then it is an update, then do some logic.
in order for this to work, i need to make sure the IBROADLOGID is incremental, i assume it is using a database sequence, but i can't find any details about it.
while checking existing records in our production database.
I have noticed that record 1238949607 created before 1238949604, but the ID is greater than the second, which doesnt seems like the id is incremental.
I am not sure if it is Adobe gets the id from sequence first, then do the insertion ? this happens because the insertion might depends on the delivery?