Expand my Community achievements bar.

Join us on September 25th for a must-attend webinar featuring Adobe Experience Maker winner Anish Raul. Discover how leading enterprises are adopting AI into their workflows securely, responsibly, and at scale.
SOLVED

Data flow run on demand

Avatar

Level 4

I would like to run on demand a data flow for a specific source. I checked this documentation: On-demand Ingestion for Sources Dataflows In The UI | Adobe Experience Platform

These are the parameters required for running on demand. 

 

Ingestion start time - The scheduled time of when the on-demand flow run will begin.
Date range start time - The earliest date and time that data will be retrieved from.
Date range end time - The date and time that data will be retrieved up to.

 

Below, what date is that which must be selected for "Date range start time" and "Date range end time" parameters? The wording in the documentation is very unclear. 

This is an SFTP source, and the data resides in a certain csv file in the location (like MyFile.csv). The file is periodically overwritten by keeping the same name, and I want to import the new content, in full.

 

Any clarification will be highly appreciated.

1 Accepted Solution

Avatar

Correct answer by
Level 5

In AEP,  "Date range start time" and "end time" for on-demand ingestion refer to the file's last modified timestamp on the SFTP server — not the data inside the file.

Since your file (MyFile.csv) is periodically overwritten with the same name:

1) Set a date range that includes the file’s last modified time to ensure it's picked up.

2) If unsure, use a broad time window (e.g., last 1–2 days) to ensure inclusion.

For regular ingestion of an updated file, consider creating a recurring dataflow instead of using on-demand.

View solution in original post

3 Replies

Avatar

Correct answer by
Level 5

In AEP,  "Date range start time" and "end time" for on-demand ingestion refer to the file's last modified timestamp on the SFTP server — not the data inside the file.

Since your file (MyFile.csv) is periodically overwritten with the same name:

1) Set a date range that includes the file’s last modified time to ensure it's picked up.

2) If unsure, use a broad time window (e.g., last 1–2 days) to ensure inclusion.

For regular ingestion of an updated file, consider creating a recurring dataflow instead of using on-demand.

Avatar

Level 4

Thanks Kumar. I will try this. And for regular ingestion, yes, I am using a recurring flow. I use the run on demand just for out-of-the-ordinary run, when we need to push data into the Platform outside the regular schedule.

Avatar

Administrator

@GabrielaNa1 Just checking in — were you able to resolve your issue?
We’d love to hear how things worked out. If the suggestions above helped, marking a response as correct can guide others with similar questions. And if you found another solution, feel free to share it — your insights could really benefit the community. Thanks again for being part of the conversation!



Kautuk Sahni