AEM author extreme slowness after publish/asset move operation

[helloosuman]

AEM 6.5 SP 17 / AEM Guides 4.3.1.5 / RHEL8 / Java 11

AEM author extreme slowness after publish/asset move operation. An ditamap with 25 topics gets published in 45min-1hr or, asset gets moved after 3-4 hour when we observe below errors: 

08.08.2024 15:52:36.326 *WARN* [sling-threadpool-3e334b00-3be5-4a91-a4b8-7aec0360882e-(apache-sling-job-thread-pool)-8-Reference Update Processing Queue(ref-updater/delete)] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 190000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [nt:base] as a where contains(*, '"/content/dam/mygps/temp/fmTempDitamaps"') /* xpath: //*[jcr:contains(., '"/content/dam/mygps/temp/fmTempDitamaps"')] */ fullText="/content/dam/mygps/temp/fmTempDitamaps", path=*)
08.08.2024 15:52:36.331 *WARN* [sling-threadpool-3e334b00-3be5-4a91-a4b8-7aec0360882e-(apache-sling-job-thread-pool)-8-Reference Update Processing Queue(ref-updater/delete)] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 200000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [nt:base] as a where contains(*, '"/content/dam/mygps/temp/fmTempDitamaps"') /* xpath: //*[jcr:contains(., '"/content/dam/mygps/temp/fmTempDitamaps"')] */ fullText="/content/dam/mygps/temp/fmTempDitamaps", path=*)
08.08.2024 15:52:36.522 *INFO* [10.8.27.13 [1723146756504] POST /bin/validatexml HTTP/1.1] com.adobe.fmdita.profiles.CatalogResolver Unable to resolve pId -//W3C//ENTITIES MathML 3.0 Qualified Names 1.0//EN systemId mathml3-qname.mod
08.08.2024 15:52:36.692 *WARN* [10.8.27.13 [1723146756691] GET /bin/guides/translation/languagevariable HTTP/1.1] com.adobe.fmdita.xmltranslation.service.TranslationLanguageVariableService No matching file for language code : en found inside /libs/fmdita/translation/lngvar
08.08.2024 15:52:38.195 *WARN* [sling-threadpool-3e334b00-3be5-4a91-a4b8-7aec0360882e-(apache-sling-job-thread-pool)-8-Reference Update Processing Queue(ref-updater/delete)] org.apache.jackrabbit.oak.query.FilterIterators The query read or traversed more than 200000 nodes.
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 200000 nodes. To avoid affecting other tasks, processing was stopped.
08.08.2024 15:52:38.195 *ERROR* [sling-threadpool-3e334b00-3be5-4a91-a4b8-7aec0360882e-(apache-sling-job-thread-pool)-8-Reference Update Processing Queue(ref-updater/delete)] org.apache.sling.jcr.resource.internal.helper.jcr.JcrNodeResourceIterator seek: Problem creating Resource for next node, skipping
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 200000 nodes. To avoid affecting other tasks, processing was stopped.

at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70) [org.apache.jackrabbit.oak-core:1.22.15]

at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67) [org.apache.jackrabbit.oak-core:1.22.15]

at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411) [org.apache.jackrabbit.oak-lucene:1.22.15]

at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392) [org.apache.jackrabbit.oak-lucene:1.22.15]

at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]

at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]

at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]

at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216) [org.apache.jackrabbit.oak-core:1.22.15]

at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432) [org.apache.jackrabbit.oak-lucene:1.22.15]

at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515) [org.apache.jackrabbit.oak-core:1.22.15]

[EstebanBustamante]

Hi, 

The errors indicate that full-text queries are traversing too many nodes and there are issues with resolving certain resources and files. This causes the queries to stop and the process to potentially fail. I suggest the following two strategies:

- Upgrade the indexes so that the queries can be resolved more efficiently with a custom query. [1]

- Examine the content tree structure to ensure it is correctly organized. A folder with a large number of nodes can significantly impact server performance. [2]

Please check this for more info: 

[1]. https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/operations/query... 

https://sourcedcode.com/blog/aem/optimizing-debugging-search-queries-explain-query-tool-aem 

[2]. https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/maximum-number-of-nodes-on... 

Hope this helps

[kautuk_sahni]

@helloosuman Did you find the suggestions from users helpful? Please let us know if you require more information. Otherwise, please mark the answer as correct for posterity. If you've discovered a solution yourself, we would appreciate it if you could share it with the community. Thank you!

[helloosuman]

Nope ..that suggestion didn't help. Please note that we have engaged adobe production support team from AEM (our's being platinum member) to look into and even they haven't found resolution yet.