Check if value of file is already in table (Schema)

[god_prophet]

We need to valudate if columna A has already a value inserted into a schema, if so, workflow should end.

Is it possible?

For example: if at 9 am file myfile.csv has value: "DE1" in columna "A"... workflow should continue.

If at 11 am, the same file (that maybe was replaced but has the same name) has the value "DE1" in column "A", the process should end because all the rows were already ingested in the previous run. however, if the columna "A" has other value like "DE2", the workflow should import all the rows in the file.

[Heku_]

Hello, you should use a fork and an enrichment after your data loading file activity, making the enrichment with your table by the column "A", after that put a test that checks if there are some records (vars.recCount > 0):

- If there are some records -> Go to End

- Else: Recover the data from the data loading and process it

Hope it helps.

Heku