"Failed to create thread" while calling API thousands of times

Question

Hello,I have a simple workflow that is failing on the JS Code box.I am getting "Failed to create thread" when I try to process more than 18,000 recipients.  Here is what the wokflow does: It targets recipients, add empty columns and populate these columns by calling an external API.In the javascript code, I am basically looping over all the targets and make an API call with this function : function processResponse(response, userid){
    var response = response[1];
    response = JSON.parse(response);

// ... parse JSON reponsive and collect data

var query = "UPDATE ....";
// ... update empty columns in temporary table with retrieved data

sqlExec(query);
}

function callService(userid) {
    var command = "wget -qO- --header 'Authorization: Basic XXXX' https://example.com/contacts/"+userid;
    var response = execCommand(command);
    processResponse(response, userid);
}

for each(var usertId in sqlSelectUserIds) {
  callService(usertId[0]);
}

cnx.dispose();   Do you know what's the problem? Thank you for your help. R

Darren_Bowers · Accepted Answer

Hi @roro_coeur - This is a known issue with JS activities. You only have a finite amount of memory available to execute the javascript and depending on how much processing you want to do, you can run out of available memory to do processing or to spawn execCommand processes like you found. I would see if there is a way you can batch these external requests rather than doing individual requests.

Another option is to only process 10k records at a time in your JS activity and then loop around again in the workflow back to the same JS activity and then process the next lot of records. This will hopefully reset the available memory available to the JS activity.

Cheers Darren

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded