Process data received by External Signal | Community
Skip to main content
D
dipendu_g
Level 5
April 7, 2023
Solved

Process data received by External Signal

  • April 7, 2023
  • 1 reply
  • 1909 views

Hello All,

 

We have a common workflow with an External signal at the start.

This workflow is called multiple times by various other workflows, sometimes 15/20 per minute.

 

We want to process each request 1 by 1, rather than in bulk. i.e When one request gets completed the next request should be picked up. I understand that External Signal queue's up the request, is there a way to handle this one at a time, any reference would really help

 

 

Regards,

DG

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Marcel_Szimonisz

Hello @dipendu_g ,

basically each time you call the workflow with the signal activity in it, will create new workflow, adds your input data params to it and will run it.

If you call signal activity 10 times it will create 10 workflows

You will need to:

  • create template that contains all activities after you signal activity.
    But instead of signal activity you put start change the name to eg "start00". Set the execution folder somewhere where all the workflows will be dumped so you have them nicelly separated. Save the template as "someNameWorkflow"
  • in the original workflow  you are only left with signal activity. Where you send data from somwhere right?
    Go into the intiialization script and add following code
var initVars = "vars.myvar1='" + vars.myvar1 + "'\n;"+ "vars.myvar2='" + vars.myvar1 + "'\n;"; var workflowVars = <workflow label={"Some nice name " + formatDate(new Date,"- %4Y%2M%2D%02H:%02N:%02S")} <activities> <start name="start00"> <initScript>{initVars}</initScript> </start> </activities> </workflow>; NLWS.xtkWorkflow.spawn( 'someNameWorkflow', <workflow> <activities> <start name="start00"> <initScript></initScrip> </extern> </activities> </workflow> )​
  • then when finished and you send signal to the wokflow with the signal activity it will create new workflow  everytime. After some time you will end up with thousands of workflows. Ideally is to create clean up service. It will be another workflow sheduled to run every let's say 30m to remove all workflows from the folder that are finished and last start time is more than 30m. then simply we call update data activity with delete option. THis will only keep all failed workflows for you to fix and restart. and worfklows that are not older than 30m

 

Marcel

1 reply

Marcel_Szimonisz
Community Advisor
Marcel_SzimoniszCommunity Advisor
Community Advisor
April 7, 2023

Hello @dipendu_g,

you want to speed up the through put over this workflow right?

I would spawn the workflow, based on template which you create out of everything after the signal activity, each time request comes. Signal activity initialization script will spawn new workflow, which will process the signal data input. This way the queue wait time will decrease.

 

Additionally you can then create clean up workflow which will clean up everything older than hour and finished.

 

 

Marcel Szimonisz

MarTech Consultant
for more tips visit my blog
https://www.martechnotes.com/

D
dipendu_gAuthor
Level 5
April 7, 2023

Hi @marcel_szimonisz,

 

Any example workflow, that I can have a look, with the code.

 

 

Can you please elaborate on this part "Additionally you can then create clean up workflow which will clean up everything older than hour and finished."

 

Any working example or tidbits might help me to start

 

Regards.

DG

Marcel_Szimonisz
Community Advisor
Marcel_SzimoniszCommunity AdvisorAccepted solution
Community Advisor
April 7, 2023

Hello @dipendu_g ,

basically each time you call the workflow with the signal activity in it, will create new workflow, adds your input data params to it and will run it.

If you call signal activity 10 times it will create 10 workflows

You will need to:

  • create template that contains all activities after you signal activity.
    But instead of signal activity you put start change the name to eg "start00". Set the execution folder somewhere where all the workflows will be dumped so you have them nicelly separated. Save the template as "someNameWorkflow"
  • in the original workflow  you are only left with signal activity. Where you send data from somwhere right?
    Go into the intiialization script and add following code
var initVars = "vars.myvar1='" + vars.myvar1 + "'\n;"+ "vars.myvar2='" + vars.myvar1 + "'\n;"; var workflowVars = <workflow label={"Some nice name " + formatDate(new Date,"- %4Y%2M%2D%02H:%02N:%02S")} <activities> <start name="start00"> <initScript>{initVars}</initScript> </start> </activities> </workflow>; NLWS.xtkWorkflow.spawn( 'someNameWorkflow', <workflow> <activities> <start name="start00"> <initScript></initScrip> </extern> </activities> </workflow> )​
  • then when finished and you send signal to the wokflow with the signal activity it will create new workflow  everytime. After some time you will end up with thousands of workflows. Ideally is to create clean up service. It will be another workflow sheduled to run every let's say 30m to remove all workflows from the folder that are finished and last start time is more than 30m. then simply we call update data activity with delete option. THis will only keep all failed workflows for you to fix and restart. and worfklows that are not older than 30m

 

Marcel

Terms & ConditionsAccessibility statement

Loading footer...