Expand my Community achievements bar.

The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.

Avatar

Level 4
Level 4
12/6/21 6:32:24 AM

Hi There,
we want to improve the search functionality in AEM, we have been experiencing timeouts on search. 


We have even created an index for that query but seems to be not working. 

<myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
        <cq:Page jcr:primaryType="nt:unstructured">
            <properties jcr:primaryType="nt:unstructured"/>
        </cq:Page>
    </indexRules>
</myIndex>

This is the query - select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') 


We can't increase the QueryLimitReads parameter in AEM. 



Screenshot 2021-12-06 at 16.23.29.png

Views

6.9K

Replies

9
Sign in to like this content

Total Likes

Translate
Translate
9 Replies

Avatar

Community Advisor
Community Advisor
12/6/21 8:21:12 AM

Hi,

Can you try with small tree to exclude dam and XF

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content/yourproject/root')

 



Arun Patidar

Views

6.9K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
12/6/21 9:14:58 AM

Is it possible to run the queries to sectioned content limited to your project rather than the whole content root? Usually there would be a lot of nodes under /content other than just the page content like dam, experience-fragments, user-generated if any..

 

Views

6.8K

Replies

3
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
12/7/21 1:13:18 PM
In response to anelem1760873

Thanks. So when you enter "tax" in the network console, what is the request and response?
I see below for me 

 

http://localhost:4502/libs/granite/omnisearch?p.guessTotal=1000&fulltext=tax&savedSearches%40Delete=&savedSearches=&type=cq%3APage&3_path=&1000_group.p.or=true&jcr%3Acontent%2Fcq%3Atags%40TypeHint=String%5B%5D&jcr%3Acontent%2Fcq%3Atags%40Delete=&orderby=path&320_daterange.lowerBound=&320_daterange.lowerBound%40TypeHint=Date&320_daterange.upperBound=&320_daterange.upperBound%40TypeHint=Date&320_daterange.property=jcr%3Acontent%2Fcq%3AlastModified&320_daterange.breadcrumbs=jcr%3Acontent%2Fcq%3AlastModified+%7B%24lowerBound%7D+-+%7B%24upperBound%7D&6_property=jcr%3Acontent%2Fcq%3AlastReplicationAction&6_property.breadcrumbs=Publish+Status&7_property=jcr%3Acontent%2Fcq%3AlastRolledoutBy&7_property.breadcrumbs=LiveCopy+Status&contentPath=%2Fcontent&location=site&location.suggestion=Sites

And the response seems to have 10 results. Trying to understand and making sure if this is the query that is being complained on. 

Views

6.8K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Employee Advisor
Employee Advisor
12/7/21 12:48:17 PM

When you check the logs, you will find a lot of warnings about this traversal. Can you check if you get messages like "traversed more than xxx nodes" or "index-traversed more than xxx nodes"?

 

The query itself looks sound.

 

Views

6.8K

Replies

2
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 4
Level 4
12/8/21 12:28:12 AM
In response to Jörg_Hoh

This is whats showing on the logs 

08.12.2021 08:20:09.171 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:09.977 *INFO* [] [] [qtp507980041-260256] org.apache.sling.auth.core.impl.SlingAuthenticator handleLoginFailure: Unable to authenticate null: Invalid token credentials.
08.12.2021 08:20:12.492 *INFO* [] [] [qtp507980041-260195] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:18.276 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:21.054 *INFO* [] [] [qtp507980041-259766] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:26.390 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Failed to wrap datasource for lookahead
org.apache.sling.api.scripting.ScriptEvaluationException: 
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:416)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.service(DefaultSlingScript.java:491)
	at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.sling.api.SlingException: 
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspExceptionInternal(JspServletWrapper.java:691)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:608)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:533)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	... 362 common frames omitted
Caused by: org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	... 367 common frames omitted
08.12.2021 08:20:26.390 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Fallback to non-lookahead datasource
08.12.2021 08:20:27.788 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 10000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:42.668 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:44.816 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] com.adobe.granite.repository Service [88135, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
08.12.2021 08:20:44.818 *ERROR* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)



Views

6.8K

Replies

1
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Employee Advisor
Employee Advisor
12/15/21 10:37:18 AM
In response to anelem1760873

Hm, unfortunately the exception is not precise enough. What is your AEM version and what version of Oak is used (you should see it quite prominently in CRX/DE, but also the bundle versions are sufficient)? Given some details of the stacktrace I believe it's because of index-traversal. 

 

But this stacktrace is coming from a different query, because Omnisearch does not produce queries like

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') 

 

 

 

 

 

 

Views

6.8K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate
Related Conversations
Unable to find users , user roles , company information through Mangento Rest API in postman

Views

173

Likes

0

Replies

2
Adobe Developers Live, November 2024, Session | Bringing AI and Personalization to AEM Edge Delivery Services

Views

247

Likes

0

Replies

0
Adobe Developers Live, November 2024, On-Demand Session | Accelerate back-office integrations with Adobe Commerce with the Integration Starter Kit

Views

165

Likes

0

Replies

0
Adobe Developers Live, November 2024, Session | A Real-World Journey to Edge Delivery Services

Views

152

Likes

0

Replies

0
Adobe Developers Live, November 2024, Session | Forms on the Edge!

Views

131

Likes

0

Replies

0