Get the number of running workflows and Hold the workflow until the count reaches a specific number and wait for workflows to complete

[akhilraj]

Hi All,

We have a requirement to trigger workflows for multiple AEM pages.

The number of pages will be more than 1000 and based on some node operation we are triggering workflow.

Question is:

How to get the number of active workflow instance(Workflows currently running)

How we can make the workflow triggering sleep if number of active workflows reaches a specific number and run after all completed.

Any idea?

[aanchal-sikka]

Hello @akhilraj 

OOTB AEM has the capability to configure Max parallel jobs for concurrent workflows. It also one of the recommended best practices for Workflows.

https://experienceleague.adobe.com/docs/experience-manager-65/developing/extending-aem/extending-wor...

Please check, if it helps

Also, for bulk workflows, use https://adobe-consulting-services.github.io/acs-aem-commons/features/bulk-workflow-manager/index.htm... for On-premise instances

[akhilraj]

Hi @aanchal-sikka , @EstebanBustamante ,

Thanks for the response.

I think bulk workflow we may not use in our case, as we are checking lot of conditions before triggering a workflow.

For Ex:  it should not be an image, it should be a page, and activated date should be greater than modified and page should be updated with some specific properties.

So I am planning to trigger workflow on each pages which satisfy the above conditions, but there can be lot of pages, so I think, just running workflow with some thread.sleep also will cause more load on instances.

So we are thinking to get current WF instances and based on that count we can either hold the workflow or proceed

[aanchal-sikka · Accepted solution]

@akhilraj 

In my experience Thread.sleep() can cause problems, especially if system overloads due to any execution.

You can probably create a schedular or JMX bean.

Where you get 1000 nodes matching the criterion and trigger workflows. Leave onto AEM to deal via max parallel jobs (page workflows are generally not resource intensive). Mark these pages for execution complete.

refer to link below for the APIs available to get workflow status info. http://localhost:4502/system/console/jmx/com.adobe.granite.workflow%3Atype%3DMaintenance,

These would be very close to what information we can gather via APIs OOTB

[EstebanBustamante]

I agree with @aanchal-sikka ,Thread.sleep() is not a good approach.

@akhilraj I still think you could use the Bulk Workflow, this takes as an argument a query, so you can create a query that satisfied your "conditions" and If that is still hard to do, you can create a custom WF step to validate all the conditions, add this new WF step in a new workflow model and use that new workflow model to launch with the Bulk Workflow Manager, the great of this, is that all the processing, threads and failures/errors are already handled, if you go with your own implementation it can be time-consuming. Just my thought.  

[EstebanBustamante]

If you are on AEM on-premise, I would recommend using the Bulk Workflow Manager from ACS Commons, this tool can launch a specific set of workflows, it performs pretty well for huge operations.

[ManviSharma]

Hi,

1. To get the number of active workflow instances in AEM, use the WorkflowService API and retrieve the count of active workflows.
2. To make workflow triggering sleep if the number of active workflows reaches a specific number and run after completion, implement a loop that checks the active workflow count and introduce a sleep mechanism before triggering new workflows. Resume the workflow triggering process when the active workflow count drops below the desired number.

[Jörg_Hoh]

That approach will definitely lead to problems. Imagine what happens when your AEM instance is restarted sometime in between. The workflows are persisted but your "wait for condition X" code is aborted and probably won't restart.

Why can't you build a workflow package with all the payload paths, and then start a single workflow instance on that workflow package? Then you have just a single workflow, which deals with all paths, and you don't need to build a throttling solution by yourself.
And it's persistent, so a restart of the instance might just cause your workflow to start over, but not to loose any data.