I want to write a query to traverse through more than 100000(1 lac) nodes

[chinmayis865517]

I want to write a query for a groovyscript  to traverse through more than 100000(1 lac) nodes without affecting the performance.

[ayush-804]

Hi @chinmayis865517 ,

To efficiently traverse 100,000+ nodes in a Groovy script, ensure relevant properties are indexed. You can create a custom index under in CRXDE. To verify if a query is using an index, check in the Query Debugger tool. 

For best practices, please refer to:

• https://experienceleague.adobe.com/docs/experience-manager-65/deploying/deploying/queries-and-indexi...
• https://experienceleague.adobe.com/docs/experience-manager-65/deploying/practices/best-practices-for...

Regards,

Ayush

[Shiv_Prakash_Patel · Accepted solution]

Hi @chinmayis865517 ,

Please check these similar threads.

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/the-query-read-or-traverse... 

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/the-query-read-or-traverse... 

Regards

[DPrakashRaj]

best solution is to write an optimize query with proper indexing as suggested by others but iof its one time and you can't create a proper indexing you can try increasing the querylimit as well as explained on https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/what-is-maximum-limit-of-h...

[giuseppebag]

For traversing 100,000+ nodes efficiently in a Groovy script, I recommend:

1. Use ResourceResolver.getResourceResolver() instead of JCR API when possible
2. Implement batch processing with chunking (process 1000 nodes at a time)
3. Use JCR-SQL2 queries with appropriate indexing: Make sure your query has proper Oak index coverage to avoid traversal warnings in logs
4. Consider async processing with Sling Jobs for very large operations