Expand my Community achievements bar.

July 31st AEM Gems Webinar: Elevate your AEM development to master the integration of private GitHub repositories within AEM Cloud Manager.

Master instance keep's stalling

Avatar

Former Community Member

Hello all our team has run into a major problem here.

Our main instance is not working, we've tried restarting it and it will work for a few minutes then eventually stall.

We did some troubleshooting and found that there are 22000 running workflows on our offloading servers (we have 2 so 11000 each).

I was thinking to do a curl and delete the etc/workflow/instances/server0 node from the back end hoping it will solve our problem.

Is this a viable method?

Also this is what the error.log keeps spitting out.

13.07.2016 11:46:42.636 *ERROR* [sling-default-3042-org.apache.sling.event.jobs.Job:ad251b44-a812-4e15-8a45-c51e155bd723-0] org.apache.sling.commons.scheduler.impl.QuartzScheduler Exception during job execution of org.apache.sling.event.impl.jobs.scheduling.JobSchedulerImpl@6a09566e : null
java.lang.NullPointerException: null
    at org.apache.sling.event.impl.jobs.JobManagerImpl.addJobInteral(JobManagerImpl.java:537)
    at org.apache.sling.event.impl.jobs.JobManagerImpl.addJob(JobManagerImpl.java:704)
    at org.apache.sling.event.impl.jobs.JobManagerImpl.addJob(JobManagerImpl.java:267)
    at org.apache.sling.event.impl.jobs.scheduling.JobSchedulerImpl.execute(JobSchedulerImpl.java:280)
    at org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:116)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

1 Reply

Avatar

Level 5

***UPDATE***

 

So we've cleared /etc/workflows/instance/server0 node

Now we keep getting this exception

15.07.2016 13:50:17.725 *WARN* [Adobe Granite Offloading: Job Offloader] org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl Fixing non-matching id from :offloadingInput to var/eventing/offloading/packages/var/eventing/offloading/assigned/436c9c0a-e1b6-4f70-b466-e4f32c055bc9/com.adobe.granite.workflow.offloading/2016/6/7/19/16/c02fd918-0ab3-417d-974f-ab789fa823a9_393637:offloadingInput.