Level 3

Solved

Too many dam nodes need to be traversed

Forum|Forum|3 years ago
March 17, 2022
5 replies
6842 views

I hvave more than 800000 images in the dam. When I publish or unpublish a page, AEM will auto find the reference imges for this page, it will take a lot of time. This caue a page successfully be published will take serveral minutes.

Does anyone know how can I reduce the search time?

Where and how can I create a index for the query of the follow log.

POST /bin/wcmcommand HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.Cursors$TraversingCursor Traversed 810000 nodes with filter Filter(query=select [jcr:path], [jcr:score], * from [dam:Asset] as a where [jcr:content/contentFragment] = 'true' and contains(*, '/content/demo/test') and isdescendantnode(a, '/content/dam') order by [jcr:created] desc option(index tag [visualSimilaritySearch]) /* xpath: /jcr:root/content/dam//element(*, dam:Asset)[(jcr:content/@contentFragment = 'true' and jcr:contains(., '/content/demo/test'))] order by :created descending option (index tag visualSimilaritySearch) */ fullText="/content/demo/test", path=/content/dam//*, property=[:indexTag=[visualSimilaritySearch], jcr:content/contentFragment=[true]]); consider creating an index or changing the query

AEM : 6.5

Service Package : 6.5.11

Thanks,

Forrest

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by Alisahali

Hello there,

When trying to move/delete/publish either Content Fragments or Sites/Pages, there is an issue when Content Fragment references are fetched, as the background query will fail; i.e. the functionality will not work.
To ensure correct operation you need to add the following properties to the index definition node /oak:index/damAssetLucene (no re-indexing is required) :

"tags": [
"visualSimilaritySearch"
]
"refresh": true

Hope that helps. Thanks!

lukasz-m

Community Advisor

Hi @forrestli, in general there are really good guidance about search indexes. This could be good starting point for you.

Looking on the information form the log you have shared, I was able to quickly generate below index rules, using Oak Index Definition Generator, you can try to use it.

<?xml version="1.0" encoding="UTF-8"?><jcr:root xmlns:dam="http://www.day.com/dam/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0">
  <myIndex compatVersion="{Long}2" async="async" jcr:primaryType="oak:QueryIndexDefinition" evaluatePathRestrictions="{Boolean}true" type="lucene">
    <indexRules jcr:primaryType="nt:unstructured">
      <dam:Asset jcr:primaryType="nt:unstructured">
        <properties jcr:primaryType="nt:unstructured">
          <contentFragment name="jcr:content/contentFragment" propertyIndex="{Boolean}true" jcr:primaryType="nt:unstructured"/>
          <created name="jcr:created" ordered="{Boolean}true" jcr:primaryType="nt:unstructured"/>
        </properties>
      </dam:Asset>
    </indexRules>
  </myIndex>
</jcr:root>

Alternatively, if this query is related to OOTB AEM functionality, you can also contact Adobe Support team to provide package with proper index definition or extension to existing one.

Please also keep in mind one important thing, you should avoid creating new index if existing can be extended.

F

ForrestLiAuthor

Level 3

I add the property "contentFragment" under the damAssetLucene index. It is show me "The index definition has changed without reindexing." How can I make this new propery to work?

A

Alisahali

Level 3

Adding that property might be helpful but not the solution. Add properties tags and refresh to the damAssetLucene node as given below:

"tags": [
"visualSimilaritySearch"
]
"refresh": true

Note that tags is a multi String and refresh is a boolean value. Run the query tool right after and you'll notice that it'll now use damAssetLucene index.

joerghoh

Adobe Employee

Can you run that query on the queryPerformance tool [1]? This query should use the damAssetLucene index.

And then please raise a support ticket with Adobe support and reference this thread.

thanks,

Jörg

[1] https://experienceleague.adobe.com/docs/experience-manager-65/administering/operations/operations-dashboard.html?lang=en#explain-query

F

ForrestLiAuthor

Level 3

I run the query on the queryPerformance tool, it is showing "No indexes were used. This is a traversal query."

A

Alisahali

Level 3

Hey Forrestli, I ran into the same problem this morning and resolved it by adding tags and refresh properties to damAssetLucene index. Note that reindexing is not required after you add them. Your query should start using damAssetLucene right after. Also, it is a known issue with one of the 6.5 service pack. Follow the answer that I posted earlier. Thanks!

A

AlisahaliAccepted solution

Level 3

Hello there,

When trying to move/delete/publish either Content Fragments or Sites/Pages, there is an issue when Content Fragment references are fetched, as the background query will fail; i.e. the functionality will not work.
To ensure correct operation you need to add the following properties to the index definition node /oak:index/damAssetLucene (no re-indexing is required) :

"tags": [
"visualSimilaritySearch"
]
"refresh": true

Hope that helps. Thanks!

F

ForrestLiAuthor

Level 3

Hi Alisahali, Is the new node like this?

A

Alisahali

Level 3

No, you don't need a new node. Just add them as new properties directly to the damAssetLucene node please. Sorry for the confusion. For example - it should be at the same level as the reindex property if you were to perform reindexing i.e. directly against the damAssetLucene node and not the 'property' node under it.