Error Handling - storing errors in multiple external calls to be able to return in the webhook response

[VictorToledo_]

Hi all,

I have a flow where I'm iterating over a list of file names that I need to get from an S3. The flow is working fine, but now I want to add better error handling and be able to return the list of files that were successfully fetched and a list of those that had errors.

I have the Iterate module, then the S3 and in the error handling I'm accumulating in a variable the names of files that fail, but in the webhook response I can't use this value.
I mean, I don't want to add a webhook response after the errors, because the flow continues, I want to be able to use the general webook response and make reference to this variable to return it in the flow response.

i want to have something

response status 200

{

"errors" : ["file1", "file5"],

"success": ["file2","file3","file4]

}

thanks in advance!

[Sven-iX]

Hi @VictorToledo_ 

try this: (JSON attached)

• branch into processing files & capturing results
• processing
  • the S3 error handler sets a "err" variable and ends in an ignore so the processing continues
  • after getting the err msg we check
    • is error: create a JSON string {"file":"#####","error":"####"}
    • no error: just collect the file name {"file":"#####"}, use comma as separator
    • set the variable of the aggregated strings
• capture results
  • simple getVar > stuff into webhookResponse.

[VictorToledo_]

Hi @Sven-iX 

thank you very much for your help.

I tried something similar but i still having issues

my input json is something similar to

{"array1":[

                {"pro1":"val1","prop2":"val2","files-to-import":["image1.jpg","booking1_error.jpg"]},

                {"pro4":"val4","prop5":"val5","files-to-import":["booking2_error.jpg","image6.jpg",booking21_error.jpg]}

                {"pro23":"val23","prop25":"val25","files-to-import":[booking3_error.jpg,"image26.jpg","]},

               ]

}

So the ‘bookingX_error.jpg’ should cause the error and be returned in the response as ‘errors’. But as you can see the get error variable in the main stream is not getting all the error values from the set var in the error handling stream.

This is the get variable in the main flow (3 operations, it is wrong)

This is the Set variable in the error handling flow (4 errors, it is correct)

any input is welcome!

thanks in advance

[Sven-iX]

Hello @VictorToledo_ 

I can't tell what constitutes an error. Can you help me understand? 

The JSON has issues so I tried to clean it up - does this still represent the input? 

{"array1":[
    {"prop1":"val1","prop2":"val2","files-to-import":["image1.jpg","booking1_error.jpg"]},
    {"prop1":"val4","prop2":"val5","files-to-import":["booking2_error.jpg","image6.jpg","booking21_error.jpg"]},
    {"prop1":"val23","prop2":"val25","files-to-import":["booking3_error.jpg","image26.jpg"]}
   ]
}

[VictorToledo_]

Sorry for the confusion

yes, we can use that json as an example input to this part of the flow. Where we have a first array that is iterated to get the second array with the images to import. Suppose the "bookingX_error.jpg" do not exist in S3 then the S3 module will return an error for each one of them, in the example it would be 4 failed images.
Using the possible solution you created, in the "Get Error" module of the main flow, I only get 3. But in the "Set Error" of the error handling flow I can see all the errors.
That's the problem, that's what I try to describe with my screenshots. You will see the executions of each module.

For some reason I don't understand, the get variable doesn't get all the values correctly.

{
   "array1":[
      {
         "pro1":"val1",
         "prop2":"val2",
         "files-to-import":[
            "image1.jpg",
            "booking1_error.jpg"
         ]
      },
      {
         "pro4":"val4",
         "prop5":"val5",
         "files-to-import":[
            "booking2_error.jpg",
            "image6.jpg",
            "booking21_error.jpg"
         ]
      },
      {
         "pro23":"val23",
         "prop25":"val25",
         "files-to-import":[
            "booking3_error.jpg",
            "image26.jpg"
         ]
      }
   ]
}

[Sven-iX]

Hooo boy @VictorToledo_ 

I messed up  

try this flow: I forgot to initialize the "err" variable so it kept having a value when it shouldn't. I also used an IGNORE err handler which made the cycle die - instead of a RESUME

Let me know if this works now. 

[VictorToledo_]

Hello @Sven-iX 

Yes, I have tried, I mean using Resume, but in this case as I'm using an s3 module the Resume needs ‘fileName’ and ‘data’ to resume the stream, so it is not clear to me why use it.

I'm testing the flow with 4 files that fail and 3 that work. In this case the main flow is executed 3 times and the error one 4 times, that's why I think the problem occurs, when the reading of the variable (the get err of the main flow) is executed only 3 times, the last error value is lost.

[Sven-iX · Accepted solution]

Well - the code i sent works the way you describe - did you compare to yours? 

Ignore ends the processing. Resume continues the processing. 

You can put whatever in the fields or leave them blank: once you resume we catch the error and move on. The resume is for when you want to set values that the next module would use after an error condition.

[VictorToledo_]

with blank value i have errors as i mentioned

let me try it with any value but I think it will go into an infinite loop, it will try to restart with these wrong values and fail again and it will keep going like that

[Sven-iX]

You're still missing the setVar (initializing the error variable) that I added in the example JSON

also, you should put the upload into the success branch - that's why we fork the execution.

[VictorToledo_]

Hi @Sven-iX 

I have finally got it working thanks to your help, the key was the resume and then adding the router to count errors and success, all in the same stream was not possible.
Thank you very much!!!, I really appreciate you taking care and helping out