We have some hundreds of workflows running in parallel everyday through a schedular activity which is triggered daily. So each of these workflows having a wait activity in between the workflow like below:
The wait period is around 2 days due to which number of concurrent workflows is exceeding the limit which is specified as every workflow has a wait activity it continuously holding the execution.
So our idea is to split up the workflow into two 1. first workflow will run only up to the email activity 1 and then sends the export of recipient results to some S3 bucket. 2. the next workflow will pickup the recipient results from S3 bucket after two days and will process the next execution( that is sending the next email).
Here my problem is i have exported the results in csv format to S3 bucket sucessfully so part 1 is done. part 2: i have downloaded the recipient file from S3 bucket and when i am loading the data and sending the results to continously delivery. it is throwing the below error:
Part 1: Extracting the results and sending email and storing the result file in S3 bucket: which went sucessfully
I am downing the file with columns used as additional data in part 1 and uploading the data in part 2. i am using update data activity in order to linkup the targetting columns with recipient table and then sending an email. it should send only 4 emails but it is continously sending me emails like 20,21,22 and so on.
When i don't use the update data activity. i am getting an error like below :
if i dont use update data activity : My data loading result will be like this :
and i will feed this to continous delivery i am getting this erro:
Let me know if any ready made solution or any alternative we have for this.