Hypothesis Stuck Jobs - Setting Up monitoring, automatic cancel and automatic restart

[Ken_Qrious]

We have an issue where hypothesis' get stuck in the "execution in progress" indefinitely.  I've seen a few questions regarding this issue but so far no definitive answers.  

To mitigate the issue I would like to setup a technical workflow to automatically do the following.

• Determine if a hypothesis has been in the "execution in progress" state form more than 8 hours.  
• If a hypothesis has been running more than 8 hours
  • Stop the hypothesis execution
  • Restart the hypothesis execution

It looks like I can determine if a hypothesis has been running more than 8 hours by checking table nms:remaHypothesis and looking for jobs with an "execution in progress" status and a duration of more than 8 hour.  What I can't determine is how to stop the identified stuck hypothesis and restart it from a work flow.

Any suggestoins?

[Milan_Vucetic]

Hello @Ken_Qrious 

you should stop or kill the workflow first. Then you can restart it.

This should be done within Javascript step.

Regards,

Milan

[Ken_Qrious]

Hello @Milan_Vucetic ,

Thanks, that's what I was after, the command and examples to stop and restart.  I'll give that a try.

Cheers,

Ken

[Ken_Qrious · Accepted solution]

Milan put me on the right track but a few adjustments were need to make things right for my situation.  Here is what did the trick.

var strId = id from nms.remaHypothesis

nms.remaHypothesis.Stop(strId)
nms.remaHypothesis.Start(strId)