I have a query suppose we have a multiple files in a server and all the naming conventions are similar like :-
Now i have to load all the file for today's load but i have other certain data loading and enriching activities is taking lot of time and sometimes timing out.
So wanted to check if we can load one file at a time from the FTP activity.
I mean to say take one file (like Head -1) or the top in the server and once picked up then delete the file from the server, later it will be ready for the next file to be picked.
Can we do this in FTP activity, or any other alternates we can use to have this task accomplished.
At the time being, the ACC workflow activity Transfer File (using FTP/SFTP) can manage to stop after having transferred 1 file, through the Planning tab, with the checkbox Execution planned , then select the checkbox Stop planification as soon as 1 file has been transferred.
But you don't have control on what file is taken, based for example on a pattern on file basename, or size or whatever.
Moreover the planification being activated, it is not a precise control on what/when it is done, even if you set it to every hour or every 10 mn.
The idea is to use the method list (belonging also to ftp class) and so get list of files in a folder, then select a file and loop for transfer it, and do what you need in the loop, such as sleep or whatever.
If you select the checkbox mentioned, it should take only 1 file, but you have no control on which file, there is no place for regular expression (as there is in the file collector activity).
Otherwise the easier way is to use the checkbox in the Transfer file activity Planification tab.
I happen to test this, FTP activity is taking all the files at the same time even though i enabled the planned execution.
I wanted to have only one file (based on patterns) to be picked up.
Please let me know if you have any further updates on this.
Thanks in advance.
Hi Jean ,
Thanks for the reply.. I will test the option mentioned, I will let you know for any updates on the same.