Adobe Experience Manager Sites & More

anelem1760873 · 12/6/21

Hi There,
we want to improve the search functionality in AEM, we have been experiencing timeouts on search.

We have even created an index for that query but seems to be not working.

<myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
        <cq:Page jcr:primaryType="nt:unstructured">
            <properties jcr:primaryType="nt:unstructured"/>
        </cq:Page>
    </indexRules>
</myIndex>

This is the query - select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content')

We can't increase the QueryLimitReads parameter in AEM.

arunpatidar · 12/6/21

Hi,

Can you try with small tree to exclude dam and XF

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content/yourproject/root')

Arun Patidar

anelem1760873 · 12/7/21

the problem is I don't have control over the query.

Shubham_borole · 12/6/21

Is it possible to run the queries to sectioned content limited to your project rather than the whole content root? Usually there would be a lot of nodes under /content other than just the page content like dam, experience-fragments, user-generated if any..

anelem1760873 · 12/7/21

The answer to this is the same as the question above.

Shubham_borole · 12/7/21

Thanks. So when you enter "tax" in the network console, what is the request and response?
I see below for me

http://localhost:4502/libs/granite/omnisearch?p.guessTotal=1000&fulltext=tax&savedSearches%40Delete=&savedSearches=&type=cq%3APage&3_path=&1000_group.p.or=true&jcr%3Acontent%2Fcq%3Atags%40TypeHint=String%5B%5D&jcr%3Acontent%2Fcq%3Atags%40Delete=&orderby=path&320_daterange.lowerBound=&320_daterange.lowerBound%40TypeHint=Date&320_daterange.upperBound=&320_daterange.upperBound%40TypeHint=Date&320_daterange.property=jcr%3Acontent%2Fcq%3AlastModified&320_daterange.breadcrumbs=jcr%3Acontent%2Fcq%3AlastModified+%7B%24lowerBound%7D+-+%7B%24upperBound%7D&6_property=jcr%3Acontent%2Fcq%3AlastReplicationAction&6_property.breadcrumbs=Publish+Status&7_property=jcr%3Acontent%2Fcq%3AlastRolledoutBy&7_property.breadcrumbs=LiveCopy+Status&contentPath=%2Fcontent&location=site&location.suggestion=Sites

And the response seems to have 10 results. Trying to understand and making sure if this is the query that is being complained on.

anelem1760873 · 12/8/21

Thats correct, its the same query.

Jörg_Hoh · 12/7/21

When you check the logs, you will find a lot of warnings about this traversal. Can you check if you get messages like "traversed more than xxx nodes" or "index-traversed more than xxx nodes"?

The query itself looks sound.

anelem1760873 · 12/8/21

This is whats showing on the logs

08.12.2021 08:20:09.171 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:09.977 *INFO* [] [] [qtp507980041-260256] org.apache.sling.auth.core.impl.SlingAuthenticator handleLoginFailure: Unable to authenticate null: Invalid token credentials.
08.12.2021 08:20:12.492 *INFO* [] [] [qtp507980041-260195] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:18.276 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:21.054 *INFO* [] [] [qtp507980041-259766] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials
08.12.2021 08:20:26.390 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Failed to wrap datasource for lookahead
org.apache.sling.api.scripting.ScriptEvaluationException: 
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:416)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.service(DefaultSlingScript.java:491)
	at org.apache.sling.engine.impl.request.RequestData.service(RequestData.java:552)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.sling.api.SlingException: 
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspExceptionInternal(JspServletWrapper.java:691)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:608)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:533)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	... 362 common frames omitted
Caused by: org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	... 367 common frames omitted
08.12.2021 08:20:26.390 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] libs.granite.ui.components.coral.foundation.masonry Fallback to non-lookahead datasource
08.12.2021 08:20:27.788 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 10000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 90000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:42.668 *WARN* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor Index-Traversed 100000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content') /* xpath: /jcr:root/content//element(*, cq:Page)[(jcr:contains(., 'tax'))] */ fullText="tax", path=/content//*)
08.12.2021 08:20:44.816 *INFO* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] com.adobe.granite.repository Service [88135, [org.apache.jackrabbit.oak.api.jmx.SessionMBean]] ServiceEvent UNREGISTERING
08.12.2021 08:20:44.818 *ERROR* [] [] [10.62.4.17 [1638951494379] GET /mnt/overlay/granite/ui/content/shell/omnisearch/searchresults.html HTTP/1.1] org.apache.sling.engine.impl.SlingRequestProcessorImpl service: Uncaught SlingException
org.apache.jackrabbit.oak.query.RuntimeNodeTraversalException: The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.
	at org.apache.jackrabbit.oak.query.FilterIterators.checkReadLimit(FilterIterators.java:70)
	at org.apache.jackrabbit.oak.plugins.index.Cursors.checkReadLimit(Cursors.java:67)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:411)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor$1.next(FulltextIndex.java:392)
	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:646)
	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
	at org.apache.jackrabbit.oak.plugins.index.Cursors$PathCursor.hasNext(Cursors.java:216)
	at org.apache.jackrabbit.oak.plugins.index.search.spi.query.FulltextIndex$FulltextPathCursor.hasNext(FulltextIndex.java:432)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.nextInternal(SelectorImpl.java:515)
	at org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:508)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:876)
	at org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:903)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.fetch(QueryResultImpl.java:103)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:128)
	at org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$1.next(QueryResultImpl.java:83)
	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:702)
	at org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:88)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
	at org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
	at org.apache.jackrabbit.commons.iterator.RowIteratorAdapter.nextRow(RowIteratorAdapter.java:76)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.fetchNext(QueryImpl.java:854)
	at com.day.cq.search.impl.builder.QueryImpl$FilteringRowIterator.hasNext(QueryImpl.java:880)
	at com.day.cq.search.impl.builder.QueryImpl.execute(QueryImpl.java:584)
	at com.day.cq.search.impl.builder.QueryImpl.getResult(QueryImpl.java:110)
	at com.day.cq.wcm.core.impl.omnisearch.SiteOmniSearchHandler.getResults(SiteOmniSearchHandler.java:120)
	at com.adobe.granite.omnisearch.impl.core.OmniSearchServiceImpl.getSearchResults(OmniSearchServiceImpl.java:143)
	at org.apache.jsp.libs.granite.ui.components.shell.omnisearch.datasource.searchdatasource.searchdatasource_jsp._jspService(searchdatasource_jsp.java:157)
	at org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:502)
	at org.apache.sling.scripting.jsp.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:449)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.callJsp(JspScriptEngineFactory.java:339)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory.access$100(JspScriptEngineFactory.java:97)
	at org.apache.sling.scripting.jsp.JspScriptEngineFactory$JspScriptEngine.eval(JspScriptEngineFactory.java:600)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.call(DefaultSlingScript.java:388)
	at org.apache.sling.scripting.core.impl.DefaultSlingScript.eval(DefaultSlingScript.java:184)

Jörg_Hoh · 12/15/21

Hm, unfortunately the exception is not precise enough. What is your AEM version and what version of Oak is used (you should see it quite prominently in CRX/DE, but also the bundle versions are sufficient)? Given some details of the stacktrace I believe it's because of index-traversal.

But this stacktrace is coming from a different query, because Omnisearch does not produce queries like

select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'tax') and isdescendantnode(a, '/content')

Adobe Experience Manager Sites & More

The query read or traversed more than 100000 nodes. To avoid affecting other tasks, processing was stopped.

Arun Patidar

Learn

Documentation

Community

Support

Resources

Adobe account

Adobe