Hello,
I have a simple workflow that is failing on the JS Code box.
I am getting "Failed to create thread" when I try to process more than 18,000 recipients.
Here is what the wokflow does:
It targets recipients, add empty columns and populate these columns by calling an external API.
In the javascript code, I am basically looping over all the targets and make an API call with this function :
function processResponse(response, userid){
var response = response[1];
response = JSON.parse(response);
// ... parse JSON reponsive and collect data
var query = "UPDATE ....";
// ... update empty columns in temporary table with retrieved data
sqlExec(query);
}
function callService(userid) {
var command = "wget -qO- --header 'Authorization: Basic XXXX' https://example.com/contacts/"+userid;
var response = execCommand(command);
processResponse(response, userid);
}
for each(var usertId in sqlSelectUserIds) {
callService(usertId[0]);
}
cnx.dispose();
Do you know what's the problem?
Thank you for your help.
R
Solved! Go to Solution.
Views
Replies
Total Likes
Hi @roro_coeur - This is a known issue with JS activities. You only have a finite amount of memory available to execute the javascript and depending on how much processing you want to do, you can run out of available memory to do processing or to spawn execCommand processes like you found. I would see if there is a way you can batch these external requests rather than doing individual requests.
Another option is to only process 10k records at a time in your JS activity and then loop around again in the workflow back to the same JS activity and then process the next lot of records. This will hopefully reset the available memory available to the JS activity.
Cheers Darren
Views
Replies
Total Likes
Hi @roro_coeur - This is a known issue with JS activities. You only have a finite amount of memory available to execute the javascript and depending on how much processing you want to do, you can run out of available memory to do processing or to spawn execCommand processes like you found. I would see if there is a way you can batch these external requests rather than doing individual requests.
Another option is to only process 10k records at a time in your JS activity and then loop around again in the workflow back to the same JS activity and then process the next lot of records. This will hopefully reset the available memory available to the JS activity.
Cheers Darren
Views
Replies
Total Likes
Views
Replies
Total Likes
Views
Replies
Total Likes
Views
Likes
Replies
Views
Likes
Replies
Views
Likes
Replies
Views
Likes
Replies