AEM doesn't use OOTB Lucene Index with query

[johnagordon83]

Greetings,

I am executing the following query (xpath):

/jcr:root/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content//*[(jcr:like(@sling:resourceType, 'tnc/components/%') and @sling:resourceType != 'tnc/components/content/anchor-links' and @sling:resourceType != 'tnc/components/content/tab-container' and @sling:resourceType != 'tnc/components/element/tab-element')]

The query works fine and returns the expected results.  However, I expect it to use the OOTB lucene index: '/oak:index/ntBaseLucene', but it only does so on my local environment.  On our DEV/QA/STAGE and PROD environments, the index is not used and the query executes via traversal.

Here is the response from the 'Explain Query' tool on QA:

Parsing JCR-SQL2 statement: explain select [jcr:path], [jcr:score], * from [nt:base] as a where [sling:resourceType] like 'tnc/components/%' and [sling:resourceType] <> 'tnc/components/content/anchor-links' and [sling:resourceType] <> 'tnc/components/content/tab-container' and [sling:resourceType] <> 'tnc/components/element/tab-element' and isdescendantnode(a, '/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content')
cost using filter Filter(query=explain select [jcr:path], [jcr:score], * from [nt:base] as a where [sling:resourceType] like 'tnc/components/%' and [sling:resourceType] <> 'tnc/components/content/anchor-links' and [sling:resourceType] <> 'tnc/components/content/tab-container' and [sling:resourceType] <> 'tnc/components/element/tab-element' and isdescendantnode(a, '/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content'), path=/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content//*, property=[sling:resourceType=[(tnc/components/%.., is not null]])
cost for reference is Infinity
cost for property is Infinity
cost for nodeType is Infinity
Refreshed reader for index [Fulltext Index : /oak:index/fragments] [took 1ms]
Refreshed reader for index [Fulltext Index : /oak:index/damAssetLucene] [took 1ms]
cost for lucene-property is Infinity
cost for aggregate lucene is Infinity
looking for plans for paths : []
cost for aggregate solr is Infinity
cost for traverse is 2000.0
no proper index was found for filter Filter(query=explain select [jcr:path], [jcr:score], * from [nt:base] as a where [sling:resourceType] like 'tnc/components/%' and [sling:resourceType] <> 'tnc/components/content/anchor-links' and [sling:resourceType] <> 'tnc/components/content/tab-container' and [sling:resourceType] <> 'tnc/components/element/tab-element' and isdescendantnode(a, '/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content'), path=/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content//*, property=[sling:resourceType=[(tnc/components/%.., is not null]])
Traversal query (query without index): explain select [jcr:path], [jcr:score], * from [nt:base] as a where [sling:resourceType] like 'tnc/components/%' and [sling:resourceType] <> 'tnc/components/content/anchor-links' and [sling:resourceType] <> 'tnc/components/content/tab-container' and [sling:resourceType] <> 'tnc/components/element/tab-element' and isdescendantnode(a, '/content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content'); consider creating an index

Here is the ntBaseLucene index node from QA that I expect to be used with this query.

{
   "jcr:primaryType":"oak:QueryIndexDefinition",
   "compatVersion":2,
   "supersedes":[
      "/oak:index/cqDefaultFormFor",
      "/oak:index/fpContentType",
      "/oak:index/cqOwnerCanvasPage",
      "/oak:index/cqVariantFamily",
      "/oak:index/damStatus",
      "/oak:index/campaignpath",
      "/oak:index/slingVanityPath",
      "/oak:index/type",
      "/oak:index/contentPath",
      "/oak:index/cq:masterBuildingBlockPath",
      "/oak:index/containeeInstanceId",
      "/oak:index/cqCloudServiceConfigs",
      "/oak:index/slingAlias",
      "/oak:index/cqCugEnabled",
      "/oak:index/active",
      "/oak:index/cq:masterBuildingBlock",
      "/oak:index/cqAcUUID",
      "/oak:index/cqKeywords",
      "/oak:index/cqCloudServiceConfig",
      "/oak:index/cqConf",
      "/oak:index/cq:targetOfferId",
      "/oak:index/postId",
      "/oak:index/verb",
      "/oak:index/extensionType",
      "/oak:index/fpNodeType",
      "/oak:index/guideComponentType",
      "/oak:index/cqMaster",
      "/oak:index/cqIsCommunitySite",
      "/oak:index/deviceIdentificationMode",
      "/oak:index/lockCreated",
      "/oak:index/processingProfile",
      "/oak:index/ec-uuid",
      "/oak:index/slingResources",
      "/oak:index/slingResourceType",
      "/oak:index/subType",
      "/oak:index/slingResourceSuperType",
      "/oak:index/damS7watch",
      "/oak:index/jcrLockOwner",
      "/oak:index/cqTemplate",
      "/oak:index/fragmentPath"
   ],
   "seed":-8469734158828420701,
   "type":"lucene",
   "async":[
      "async",
      "nrt"
   ],
   "evaluatePathRestrictions":true,
   "reindex":false,
   "reindexCount":1,
   "indexRules":{
      "jcr:primaryType":"nt:unstructured",
      "nt:base":{
         "jcr:primaryType":"nt:unstructured",
         "properties":{
            "jcr:primaryType":"nt:unstructured",
            "resourceType":{
               "jcr:primaryType":"nt:unstructured",
               "valueExcludedPrefixes":["slingevent:Job","sling:Folder"],
               "name":"sling:resourceType",
               "sync":true
            }
         }
      }
   }
}

 Here is an example of a node that would be returned by the query.  It is located at: /content/tnc/nature/us/en-us/test-pages/general-content-test/jcr:content/body/image

{
   "jcr:primaryType":"nt:unstructured",
   "jcr:createdBy":"",
   "jcr:lastModifiedBy":"",
   "cvariationVideo":"medium",
   "desktopImageCrop":"left-top",
   "jcr:created":"Tue Mar 25 2025 20:03:03 GMT+0000",
   "verticalHorizontalSelection":"no",
   "mobileImageCrop":"center-middle",
   "jcr:lastModified":"Tue Mar 25 2025 20:04:14 GMT+0000",
   "sling:resourceType":"tnc/components/content/image",
   "compVariation":"img-vari",
   "backgroundImg1":"/content/dam/tnc/nature/en/",
   "cvariation":"medium"
}

Any thoughts on why the index is being ignored or how I might further debug?  Should I reindex?  Thank you!

[AmitVishwakarma · Accepted solution]

Hi @johnagordon83 ,

Step 1: Check if the Index Exists and is Enabled

     - Run this query in CRX/DE Query Debugger:

SELECT * FROM [oak:QueryIndexDefinition] WHERE [jcr:path]='/oak:index/ntBaseLucene'

     - If the index does not exist, you must create it.
     - If the index exists but is not being used, proceed to the next steps.

Step 2: Force Reindexing:
     - Go to CRX/DE Lite (/crx/de/index.jsp)
     - Navigate to /oak:index/ntBaseLucene
     - Set reindex = true
     - Save and restart AEM.
     - Check if your query now uses Lucene via the Explain Query tool.

Step 3: Modify Lucene Index for Guaranteed Matching
Modify /oak:index/ntBaseLucene to explicitly include sling:resourceType as an indexed property.

Make these changes in the index definition:
     - Open /oak:index/ntBaseLucene
     - Locate indexRules/nt:base/properties
     - Ensure the following entry exists:

"slingResourceType": {
    "name": "sling:resourceType",
    "propertyIndex": true,
    "analyzed": true
}

    - Save and restart AEM

Step 4: Confirm Index is Being Used
Use the Explain Query tool with your query. If the output still says "Traversal query", proceed to the next step.

Step 5: Create a Custom Lucene Index (If OOTB Still Fails)
If the default index still does not work, create a custom index:

1. Create a new index node at /oak:index/customSlingResourceTypeIndex

2. Add the following properties:
- jcr:primaryType = oak:QueryIndexDefinition
- type = lucene
- async = ["async", "nrt"]
- evaluatePathRestrictions = true
- reindex = true
3. Add indexRules/nt:base/properties:

"slingResourceType": {
    "name": "sling:resourceType",
    "propertyIndex": true,
    "analyzed": true
}

4. Save and restart AEM.

Regards,
Amit

[giuseppebag]

hi @johnagordon83,

Begin by ensuring that the /oak:index/ntBaseLucene index on the QA environment mirrors the configuration from your local setup. Pay close attention to:

• Index Rules: Confirm that the sling:resourceType property is included and properly configured.
• Exclusions: Ensure no unintended exclusions are present in the index that might exclude relevant nodes.
• Reindexing Policies: Check if the index is set to reindex automatically or if it requires a manual trigger after updates.

Sometimes, syntax differences (e.g., extra spaces, missing commas) can cause the index not to function as expected. Use a diff tool to compare the actual index configurations side by side.

Ensure that the data structure and node types in QA match what's present locally. Differences in node hierarchies or property values can lead to the index not matching the query criteria.

In the index’s definition review the supersedes property. If older or unrelated indexes are listed here, they could interfere with the current index’s functionality - remove or update these entries if necessary.

If configuration discrepancies are found or if the data has changed significantly, forcing a reindex can help refresh the index with current data. This can be done through CRXDE interface by setting the reindex property to true for the index.