Expand my Community achievements bar.

The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.

Avatar

Level 4

Hi There,
we want to improve the search functionality in AEM, we have been experiencing timeouts on search. 


We have even created an index for that query but seems to be not working. 

<myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
        <cq:Page jcr:primaryType="nt:unstructured">
            <properties jcr:primaryType="nt:unstructured"/>
        </cq:Page>
    </indexRules>
</myIndex>

This is the query - select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') 


We can't increase the QueryLimitReads parameter in AEM. 



Screenshot 2021-12-06 at 16.23.29.png

9 Replies

Avatar

Community Advisor

Hi,

Can you try with small tree to exclude dam and XF

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content/yourproject/root')

 



Arun Patidar

Avatar

Community Advisor

Is it possible to run the queries to sectioned content limited to your project rather than the whole content root? Usually there would be a lot of nodes under /content other than just the page content like dam, experience-fragments, user-generated if any..

 

Avatar

Community Advisor

Thanks. So when you enter "tax" in the network console, what is the request and response?
I see below for me 

 

http://localhost:4502/libs/granite/omnisearch?p.guessTotal=1000&fulltext=tax&savedSearches%40Delete=&savedSearches=&type=cq%3APage&3_path=&1000_group.p.or=true&jcr%3Acontent%2Fcq%3Atags%40TypeHint=String%5B%5D&jcr%3Acontent%2Fcq%3Atags%40Delete=&orderby=path&320_daterange.lowerBound=&320_daterange.lowerBound%40TypeHint=Date&320_daterange.upperBound=&320_daterange.upperBound%40TypeHint=Date&320_daterange.property=jcr%3Acontent%2Fcq%3AlastModified&320_daterange.breadcrumbs=jcr%3Acontent%2Fcq%3AlastModified+%7B%24lowerBound%7D+-+%7B%24upperBound%7D&6_property=jcr%3Acontent%2Fcq%3AlastReplicationAction&6_property.breadcrumbs=Publish+Status&7_property=jcr%3Acontent%2Fcq%3AlastRolledoutBy&7_property.breadcrumbs=LiveCopy+Status&contentPath=%2Fcontent&location=site&location.suggestion=Sites

And the response seems to have 10 results. Trying to understand and making sure if this is the query that is being complained on. 

Avatar

Employee Advisor

When you check the logs, you will find a lot of warnings about this traversal. Can you check if you get messages like "traversed more than xxx nodes" or "index-traversed more than xxx nodes"?

 

The query itself looks sound.

 

Avatar

Level 4

This is whats showing on the logs 

08.12.2021 08:20:09.171 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:09.977 *INFO* [] [] [qtp507980041-260256] org.apache.sling.auth.core.impl.SlingAuthenticator handleLoginFailure: Unable to authenticate null: Invalid token credentials.
08.12.2021 08:20:12.492 *INFO* [] [] [qtp507980041-260195] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:18.276 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:21.054 *INFO* [] [] [qtp507980041-259766] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:26.390 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Failed to wrap datasource for lookahead
org.apache.sling.api.scripting.ScriptEvaluationException: 
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:416)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.service(DefaultSlingScript.java:491)
	at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.sling.api.SlingException: 
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspExceptionInternal(JspServletWrapper.java:691)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:608)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:533)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	... 362 common frames omitted
Caused by: org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	... 367 common frames omitted
08.12.2021 08:20:26.390 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Fallback to non-lookahead datasource
08.12.2021 08:20:27.788 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 10000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:42.668 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:44.816 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] com.adobe.granite.repository Service [88135, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
08.12.2021 08:20:44.818 *ERROR* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)



Avatar

Employee Advisor

Hm, unfortunately the exception is not precise enough. What is your AEM version and what version of Oak is used (you should see it quite prominently in CRX/DE, but also the bundle versions are sufficient)? Given some details of the stacktrace I believe it's because of index-traversal. 

 

But this stacktrace is coming from a different query, because Omnisearch does not produce queries like

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content')