How to use multiple files in transfer file activity in workflow.

[derekw42533281]

I have 15 ish files that I need to use to update customer data in order. 

Example:

File1.txt - this is the file that was needed to be uploaded 15 days ago. 

File15.txt - this is the latest file in our ftp to needs to be uploaded last.

These files do have timestamp in the file name. PROFILE-202310200700.txt

File Transfer Activity shows all files but when it gets to load file it only loads that latest file. So I want to delete the file that gets used to update customer dat and when the workflow loops it loads the next file. 

Am I just not seeing how to do this or is this just impossible and I need to manually upload the files? I have quite a bit and it's going to take a long time due to the workflow we use to update data, it takes 15 to 30 minutes to finish. 

Thanks for the help! 

[SatheeskannaK]

@derekw42533281 Did you try with the sort option in the file transfer activity? and within the load file activity select the "delete source file after transfer" option. This will delete the files once it's transferred to make sure storage space is recovered.

[derekw42533281]

That didn't work. It only uploaded the last file and deleted the rest. 

[Anasso1337]

If you look at your complete loadfile activiy log you should see that it actually ran as many times as they were files loaded. In your case 15 times.

This is what happens when the fileTransfer activity downloads more than one file.
The rest of the workflow loops in sequence where each activity executes for each file before moving to the next one.

So in theory selecting the Alphanumerical sorting file should ensure files are loaded in the correct order.

Now for some practical use cases (transitions forks with segmentations, exclusions, unions etc..) this activity looping mode can cause data manipulation errors.

The best solution here is forcing the whole workflow to loop with the following :

• Ensure FileTransfer only download one file at a time : use dynamic file path with filename date part as an option to be created (in your case "PROFILE-"+DateFormat(GetOption('yourOptionID'), '%4Y%2M%2D')+"*.txt"
• After workflow ends, update your option to the next day, add a new file transfer Test to check if file is present using the same dynamic file path
• Check Generate a transition to process errors to end worfklow
• Connect main transition back to the start of the workflow

Hope this helps.