Expand my Community achievements bar.

Don’t miss the AEM Skill Exchange in SF on Nov 14—hear from industry leaders, learn best practices, and enhance your AEM strategy with practical tips.
Read More & Register today!

The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.

Avatar

Level 4
Level 4
12/6/21 6:32:24 AM

Hi There,
we want to improve the search functionality in AEM, we have been experiencing timeouts on search. 


We have even created an index for that query but seems to be not working. 

<myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
        <cq:Page jcr:primaryType="nt:unstructured">
            <properties jcr:primaryType="nt:unstructured"/>
        </cq:Page>
    </indexRules>
</myIndex>

This is the query - select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') 


We can't increase the QueryLimitReads parameter in AEM. 



Screenshot 2021-12-06 at 16.23.29.png

Views

6.7K

Replies

9
Sign in to like this content

Total Likes

Translate
Translate
9 Replies

Avatar

Community Advisor
Community Advisor
12/6/21 8:21:12 AM

Hi,

Can you try with small tree to exclude dam and XF

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content/yourproject/root')

 



Arun Patidar

Views

6.7K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
12/6/21 9:14:58 AM

Is it possible to run the queries to sectioned content limited to your project rather than the whole content root? Usually there would be a lot of nodes under /content other than just the page content like dam, experience-fragments, user-generated if any..

 

Views

6.7K

Replies

3
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
12/7/21 1:13:18 PM
In response to anelem1760873

Thanks. So when you enter "tax" in the network console, what is the request and response?
I see below for me 

 

http://localhost:4502/libs/granite/omnisearch?p.guessTotal=1000&fulltext=tax&savedSearches%40Delete=&savedSearches=&type=cq%3APage&3_path=&1000_group.p.or=true&jcr%3Acontent%2Fcq%3Atags%40TypeHint=String%5B%5D&jcr%3Acontent%2Fcq%3Atags%40Delete=&orderby=path&320_daterange.lowerBound=&320_daterange.lowerBound%40TypeHint=Date&320_daterange.upperBound=&320_daterange.upperBound%40TypeHint=Date&320_daterange.property=jcr%3Acontent%2Fcq%3AlastModified&320_daterange.breadcrumbs=jcr%3Acontent%2Fcq%3AlastModified+%7B%24lowerBound%7D+-+%7B%24upperBound%7D&6_property=jcr%3Acontent%2Fcq%3AlastReplicationAction&6_property.breadcrumbs=Publish+Status&7_property=jcr%3Acontent%2Fcq%3AlastRolledoutBy&7_property.breadcrumbs=LiveCopy+Status&contentPath=%2Fcontent&location=site&location.suggestion=Sites

And the response seems to have 10 results. Trying to understand and making sure if this is the query that is being complained on. 

Views

6.7K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Employee Advisor
Employee Advisor
12/7/21 12:48:17 PM

When you check the logs, you will find a lot of warnings about this traversal. Can you check if you get messages like "traversed more than xxx nodes" or "index-traversed more than xxx nodes"?

 

The query itself looks sound.

 

Views

6.7K

Replies

2
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 4
Level 4
12/8/21 12:28:12 AM
In response to Jörg_Hoh

This is whats showing on the logs 

08.12.2021 08:20:09.171 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:09.977 *INFO* [] [] [qtp507980041-260256] org.apache.sling.auth.core.impl.SlingAuthenticator handleLoginFailure: Unable to authenticate null: Invalid token credentials.
08.12.2021 08:20:12.492 *INFO* [] [] [qtp507980041-260195] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:18.276 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:21.054 *INFO* [] [] [qtp507980041-259766] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:26.390 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Failed to wrap datasource for lookahead
org.apache.sling.api.scripting.ScriptEvaluationException: 
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:416)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.service(DefaultSlingScript.java:491)
	at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.sling.api.SlingException: 
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspExceptionInternal(JspServletWrapper.java:691)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:608)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:533)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	... 362 common frames omitted
Caused by: org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	... 367 common frames omitted
08.12.2021 08:20:26.390 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Fallback to non-lookahead datasource
08.12.2021 08:20:27.788 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 10000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:42.668 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:44.816 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] com.adobe.granite.repository Service [88135, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
08.12.2021 08:20:44.818 *ERROR* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)



Views

6.7K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Employee Advisor
Employee Advisor
12/15/21 10:37:18 AM
In response to anelem1760873

Hm, unfortunately the exception is not precise enough. What is your AEM version and what version of Oak is used (you should see it quite prominently in CRX/DE, but also the bundle versions are sufficient)? Given some details of the stacktrace I believe it's because of index-traversal. 

 

But this stacktrace is coming from a different query, because Omnisearch does not produce queries like

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') 

 

 

 

 

 

 

Views

6.6K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate
Related Conversations
Adobe Developers Live, November 2024, On-Demand Session | Accelerate back-office integrations with Adobe Commerce with the Integration Starter Kit

Views

26

Likes

0

Replies

0
Adobe Developers Live, November 2024, Session | A Real-World Journey to Edge Delivery Services

Views

53

Likes

0

Replies

0
Adobe Developers Live, November 2024, Session | Forms on the Edge!

Views

53

Likes

0

Replies

0
Adobe Developers Live, November 2024, Session | UI Extensibility - Magic Buttons and how to build them.

Views

58

Likes

0

Replies

0
Where to Find skilled AEM Developers inside EU?

Views

127

Likes

0

Replies

3