file transfer activity execution order

[DishaSharma]

Hi All,

I have issue/confusion with file transfer activity

in my wkf i have file trasnfer actiivty then dataloading actiity then son on

my workflow is running every 30mins and in every 30min I'm receiving file,

BUT My ask is if my wkf run for 1st time-it will take file 1,file2,file3

now when workflow run for 2nd time how it will exclude 

file 1,file2,file3 to pick up,i mean it should not pickup fil1,file2 and file3

it shouuld pick new then files received only in last 30mins

this is confusion,

please help 

@ParthaSarathy

@ParthaSarathy 

@ParthaS 

@AkshayAnand 

@Craig_Thonis 

@Manoj_Kumar_ 

@Amine_Abedour 

@Marcel_Szimonisz

@DavidKangni 

@Amit_Kumar 

@ShashankNigam02

@Ganesh5067

@Jonathon_wodnicki

[AkshayAnand · Accepted solution]

Hi @DishaSharma 

You can use the file historization setting to archive the file that has been process by the activity. These files will be moved to an archive folder mentioned by you and then the location path will be empty for the new files to receive and process. 

Please find the reference here.

Regards

Akshay

[DishaSharma]

can u pleaase confirm:

do u mean once the wkf is completed, we need we need to use another file transfer activty nad uplaod that file to another folder on sftp and then in file historisation seeting --chck option--

delete the source file after transfer ,please confirm my und

also,is there any other u mentioned to do this and its diff from what i mentioned,

please specify excat step plsssssssssssssssssssssssssssssssss

@AkshayAnand 

[AkshayAnand]

Hi @DishaSharma 

As mentioned by @isahore, you can set these options in the same file transfer activity which you are using to collect the file at the starting of the workflow. In File hostorization setting you can mention your storage/archive folder path and after processing these files will go and sit in the archived folders.

Hope I am clear in my comments.

Thanks

Akshay

[DishaSharma]

@AkshayAnand 

i hav one dounbt here

what if file transfer activity  put the file in archived folder but after wkf continue to run,,then wkf got failed because of any error at update data step acitivty (which is after data laoding actiity)

then if i run the wkf again that partilur file,wkf  will not be able to process right becuase it is trnsfered to archive folder after fiel transfer activity step

please explain how we manage this flow

when we have import wkf which runs every 30mins and if we want only unprocessed files to be picked up

case: if wkf fail after file transfer and dataloaidng activity step

[AkshayAnand]

Hi @DishaSharma 

For the failure step, you can put a personalized alert activity passing the file name, hence when the workflow fails an alert will be triggered and you can try to pick the files from archived folder and reprocess it manually.

Or you can automate the process by writing Linux codes to transfer the unprocessed file from archived folder to correct location and send a signal to import workflow for a restart to pick the files again after rectification of the original error.

Thanks

Akshay

[DishaSharma]

@AkshayAnand 

i und this:

""For the failure step, you can put a personalized alert activity passing the file name, hence when the workflow fails an alert will be triggered and you can try to pick the files from archived folder and reprocess it manually.""

my ask is :
For the failure step, you can put a personalized alert activity passing the file name: h=

i have multiple acitivities agfter data loading and before update data acootty-

wkf can fail at any step--how and where whcih step i have to set this alert

a personalized alert activity passing the file name

[DishaSharma]

@isahore , @AkshayAnand ,

Please help to reply on this:

i have multiple acitivities agfter data loading and before update data acootty-

wkf can fail at any step--how and where whcih step i have to set this alert

============

how we can achieve this

For the failure step, you can put a personalized alert activity passing the file name, hence when the workflow fails an alert will be triggered and you can try to pick the files from archived folder and reprocess it manually.""

[DishaSharma]

@AkshayAnand??

[isahore]

@DishaSharma,

you can add an alert activity in the error transition of the activity where the file processing can fail (if at all that activity offers an error transition). Otherwise, on workflow failure you have go and look in the activity logs to check which file caused the failure, and then reprocess it from the archive directory.

[isahore]

Hi @DishaSharma,

The file transfer activity has a setting in the advanced parameters to delete the source files after transfer.

Select this option and your original files will be deleted from the location and not be processed the 2nd time.

BR,

Ishan

[DishaSharma]

@isahore 

so if we select this option --- delete the source files after transfer.

I have one ask on this---

it will delete the files from sftp right after the file transfer activityy pick the file or after the complete wkf has processed the file meaning

file will be deleted  from sftp after filetransfer acitiy step completed in workflow or after update data acity complete (this uodate data acitiy is after datalodign activity in my workflow)

[isahore]

Hi @DishaSharma,

Once the workflow activity picks up a file for processing, it will delete the original file right then from the external SFTP directory. It does not wait for the workflow to complete.

ACC implicitly copies the file into its specified archive directory, from where it uses the file for further processing. It does not depend on whether the original file was deleted or not. So, you should be good checking the option to delete original files after transfer.

Let me know if that was clear.

BR,

Ishan

[DishaSharma]

@isahore :

ACC implicitly copies the file into its specified archive directory, from where it uses the file for further processing. It does not depend on whether the original file was deleted or not. So, you should be good checking the option to delete original files after transfer.

specified archive directory,: where is this located ,

can u explain with screen**bleep**s

ACC implicitly copies the file into its specified archive directory, from where it uses the file for further processing.: what prupose of tis further processing

[isahore]

@DishaSharma,

Archive directory can be specified explicitly in the file transfer activity, by default it uses the default directory. This setting is available in the file historization settings

When you use the default setting, you can view the activity logs upon execution to check what is the default directory.

BR,

Ishan

[DishaSharma]

@isahorethanks

i hav one dounbt here

what if file transfer activity put the file in archived folder but after wkf continue to run,,then wkf got failed because of any error at update data step acitivty (which is after data laoding actiity)

then if i run the wkf again that partilur file,wkf will not be able to process right becuase it is trnsfered to archive folder after fiel transfer activity step

please explain how we manage this flow

when we have import wkf which runs every 30mins and if we want only unprocessed files to be picked up

case: if wkf fail after file transfer and dataloaidng activity step

please help