Row-level ingestion from the SnowflakeDB connector works for the initial backfill, however, subsequent incremental loads based on the delta column are not being ingested into AEP, and no updates are flowing through.
Hi Folks,
Based on the referenced document, I created a date filter to limit ingestion from a Snowflake table that contains the last 10 years of historical data. The requirement was to ingest only the last one month of data initially and then continue with incremental ingestion going forward.
The initial load completes successfully, and we can see the expected data in AEP. However, during the incremental runs, the dataflow fails to ingest any new records, even though new data is available in Snowflake.
It appears that the delta column–based incremental logic is not picking up the latest data, resulting in no records being ingested during incremental executions
PQL filter in the Source connection works as expected
!--scriptorstartfragment-->
{
"type": "PQL",
"format": "pql/json",
"value": {
"nodeType": "fnApply",
"fnName": ">=",
"params": [
{
"nodeType": "fieldLookup",
"fieldName": "business_process_dt"
},
{
"nodeType": "literal",
"value": "2026-03-01"
}
]
}
}
https://experienceleague.adobe.com/en/docs/experience-platform/sources/api-tutorials/filter
https://experienceleague.adobe.com/en/docs/experience-platform/sources/api-tutorials/collect/payments
Inside Dataflow creation the delta column definition !--scriptorstartfragment-->!--scriptorendfragment-->
{
"transformations": [
{
"name": "Copy",
"params": {
"deltaColumn": {
"name": "business_process_dt",
"dateFormat": "YYYY-MM-dd",
"timezone": "UTC"
}
}
},!--scriptorendfragment-->