Empty workflows keep generating automatically with no information

[dhfw]

Hello, 

We I'm facing an Issue where empty workflows keep generating automatically every day all day without stopping as if there is an infinite loop running somewhere but I can't seem to find where. 

The index for the worklfow table is already at "1 103 571 039".

Each time I refresh a worklfow is created and deleted and the number keeps adding up and at the end of the day, some workflows persist. 

Has anyone faced a similar issue ? 

Worklfows are without a label or content. Not the slightest information to dectect their origin.

[Marcel_Szimonisz]

Hello @dhfw ,

• stop wfserver if you can

• try to find it by eg. list all workflows that are running and XML memo contains "Spawn" or "queryDef" or "Write"

is the wf server down and still it is happening?
check logs if there is nothing unusual around web@default

Marcel

[dhfw]

Hello @Marcel_Szimonisz 

Thanks for your reply, Looking for  "Spawn" or "queryDef" or "Write" doesn't sadly bring much. There are only few workflows that create other workflows through "Write" and "Spawn" and they are both "Mgt" workflows. 

I'm sadly on production environment so I can't stop the wfserver, the issue is reproductible only on production. 

[Marcel_Szimonisz]

Hello @dhfw is it on premise? Can you restart server?
Basically you can create workflow with many ways

• from workflow
• from SOAP calls
  • from web app
  • workflow
  • from external systems
  • in forms
• on database level some process that is insterting new records

Can you check logs?

• server logs (in monitoring tab)
  • default --> web
  • instance --> wfserver

•  Database logs

[dhfw]

Hello @Marcel_Szimonisz ,

Thanks for the suggestions. 

After further searching in the wfserver log file, I think I may have found the root cause of the problem.

I've found a workflow that uses the function "nms.delivery.RecomputeStats" on a loop of thousands of deliveries. Apparently this function creates workflows and deletes them for each delivery (I'm unsure about this behavior since I couldn't find the definition of the function on the nms:delivery.js file) and I believe that this function doesn't always delete the workflows which causes the trail of worklows to remain behind.

These are currently just speculations while I keep digging further.

Please share your thoughts with me about this conclusion. 

Thank you.

[Amine_Abedour · Accepted solution]

hello @dhfw 
I have observed the same symptoms in the instances I have previously worked on. My analysis aligns with yours, as I had concluded that these ghost workflows are created during the execution of the tracking log retrieval workflow (internally name of the workflow: "tracking"). In this tracking workflow, the second argument of the executed command (-update) triggers the recalculation of diffusion statistics. This is accomplished through the RecomputeStats function.

Br,

Amine

[dhfw]

Very insightful @Amine_Abedour .

Thank you for your remark.