Level 4

The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.

Forum|Forum|4 years ago
December 6, 2021
3 replies
8909 views

Hi There, 🙂
we want to improve the search functionality in AEM, we have been experiencing timeouts on search.

We have even created an index for that query but seems to be not working.

<myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
        <cq:Page jcr:primaryType="nt:unstructured">
            <properties jcr:primaryType="nt:unstructured"/>
        </cq:Page>
    </indexRules>
</myIndex>

This is the query - select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content')

We can't increase the QueryLimitReads parameter in AEM.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

arunpatidar

Community Advisor

Hi,

Can you try with small tree to exclude dam and XF

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content/yourproject/root')

Arun Patidar

anelem1760873Author

Level 4

the problem is I don't have control over the query.

Shubham_borole

Community Advisor

Is it possible to run the queries to sectioned content limited to your project rather than the whole content root? Usually there would be a lot of nodes under /content other than just the page content like dam, experience-fragments, user-generated if any..

anelem1760873Author

Level 4

The answer to this is the same as the question above.

Shubham_borole

Community Advisor

Thanks. So when you enter "tax" in the network console, what is the request and response?
I see below for me

http://localhost:4502/libs/granite/omnisearch?p.guessTotal=1000&fulltext=tax&savedSearches%40Delete=&savedSearches=&type=cq%3APage&3_path=&1000_group.p.or=true&jcr%3Acontent%2Fcq%3Atags%40TypeHint=String%5B%5D&jcr%3Acontent%2Fcq%3Atags%40Delete=&orderby=path&320_daterange.lowerBound=&320_daterange.lowerBound%40TypeHint=Date&320_daterange.upperBound=&320_daterange.upperBound%40TypeHint=Date&320_daterange.property=jcr%3Acontent%2Fcq%3AlastModified&320_daterange.breadcrumbs=jcr%3Acontent%2Fcq%3AlastModified+%7B%24lowerBound%7D+-+%7B%24upperBound%7D&6_property=jcr%3Acontent%2Fcq%3AlastReplicationAction&6_property.breadcrumbs=Publish+Status&7_property=jcr%3Acontent%2Fcq%3AlastRolledoutBy&7_property.breadcrumbs=LiveCopy+Status&contentPath=%2Fcontent&location=site&location.suggestion=Sites

And the response seems to have 10 results. Trying to understand and making sure if this is the query that is being complained on.

joerghoh

Adobe Employee

When you check the logs, you will find a lot of warnings about this traversal. Can you check if you get messages like "traversed more than xxx nodes" or "index-traversed more than xxx nodes"?

The query itself looks sound.

anelem1760873Author

Level 4

This is whats showing on the logs

08.12.2021 08:20:09.171 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:09.977 *INFO* [] [] [qtp507980041-260256] org.apache.sling.auth.core.impl.SlingAuthenticator handleLoginFailure: Unable to authenticate null: Invalid token credentials.
08.12.2021 08:20:12.492 *INFO* [] [] [qtp507980041-260195] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:18.276 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:21.054 *INFO* [] [] [qtp507980041-259766] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:26.390 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Failed to wrap datasource for lookahead
org.apache.sling.api.scripting.ScriptEvaluationException: 
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:416)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.service(DefaultSlingScript.java:491)
	at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.sling.api.SlingException: 
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspExceptionInternal(JspServletWrapper.java:691)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:608)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:533)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	... 362 common frames omitted
Caused by: org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	... 367 common frames omitted
08.12.2021 08:20:26.390 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Fallback to non-lookahead datasource
08.12.2021 08:20:27.788 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 10000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:42.668 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:44.816 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] com.adobe.granite.repository Service [88135, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
08.12.2021 08:20:44.818 *ERROR* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)

joerghoh

Adobe Employee

Hm, unfortunately the exception is not precise enough. What is your AEM version and what version of Oak is used (you should see it quite prominently in CRX/DE, but also the bundle versions are sufficient)? Given some details of the stacktrace I believe it's because of index-traversal.

But this stacktrace is coming from a different query, because Omnisearch does not produce queries like

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content')

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded