Search Synonym

[mahamooda726898]

In Search , I am trying to implement search synonym.

I have updated Lucene and PageLucene index to have analyzer with details of synonym.

In search I have enabled search suggestion and search quick results. It is pointing to path /content/we-retail/us/en. In synonym txt file I have 2 set of entries.
1.women,womens,lady
2.men,mens,gentelmen

so during search I believe when I enter 'lady' it supposed to bring all records based on synonym matching will bring the result. But this is not happening. I believe on entering 'lady' it should even bring result of 'women' and 'womens' as well.

Please advise if there is anything I am still missing in configuration if any ?

Here is analyzers entry added in existing lucene  and cqPageLucene index

<lucene
jcr:primaryType="oak:QueryIndexDefinition"
async="[fulltext-async]"
codec="Lucene46"
compatVersion="{Long}2"
deprecated="{Boolean}true"
excludedPaths="[/var,/etc/replication,/etc/workflow/instances,/jcr:system]"
reindex="{Boolean}false"
reindexCount="{Long}1"
seed="{Long}2769657670289014804"
type="lucene">
<aggregates jcr:primaryType="nt:unstructured">
<nt:file jcr:primaryType="nt:unstructured">
<include0
jcr:primaryType="nt:unstructured"
path="jcr:content"/>
</nt:file>
</aggregates>
<analyzers jcr:primaryType="nt:unstructured">
<default jcr:primaryType="nt:unstructured">
<charFilters jcr:primaryType="nt:unstructured">
<HTMLStrip jcr:primaryType="nt:unstructured"/>
<Mapping jcr:primaryType="nt:unstructured"/>
</charFilters>
<filters jcr:primaryType="nt:unstructured">
<LowerCase jcr:primaryType="nt:unstructured"/>
<Synonym jcr:primaryType="nt:unstructured" synonyms="synonym.txt" ignoreCase="{Boolean}true">
<synonym.txt/>
</Synonym>
</filters>
<tokenizer jcr:primaryType="nt:unstructured" name="Classic"/>
</default>
</analyzers>
<indexRules jcr:primaryType="nt:unstructured">
<rep:Token jcr:primaryType="nt:unstructured"/>
<nt:base
jcr:primaryType="nt:unstructured"
includePropertyTypes="[String,Binary]">
<properties jcr:primaryType="nt:unstructured">
<analyticsProvider
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="analyticsProvider"/>
<analyticsSnippet
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="analyticsSnippet"/>
<hideInNav
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="hideInNav"/>
<offTime
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="offTime"/>
<onTime
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="onTime"/>
<allowedTemplates
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:allowedTemplates"/>
<childrenOrder
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:childrenOrder"/>
<cugEnabled
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:cugEnabled"/>
<cugPrincipals
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:cugPrincipals"/>
<cugRealm
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:cugRealm"/>
<designPath
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:designPath"/>
<isCancelledForChildren
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:isCancelledForChildren"/>
<isDeep
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:isDeep"/>
<lastModified
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastModified"/>
<lastModifiedBy
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastModifiedBy"/>
<lastPublished
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastPublished"/>
<lastPublishedBy
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastPublishedBy"/>
<lastReplicated
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastReplicated"/>
<lastReplicatedBy
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastReplicatedBy"/>
<lastReplicationAction
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastReplicationAction"/>
<lastReplicationStatus
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastReplicationStatus"/>
<lastRolledout
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastRolledout"/>
<lastRolledoutBy
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:lastRolledoutBy"/>
<name
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:name"/>
<parentPath
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:parentPath"/>
<siblingOrder
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:siblingOrder"/>
<template
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:template"/>
<trigger
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:trigger"/>
<versionComment
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="cq:versionComment"/>
<createdBy
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="jcr:createdBy"/>
<lastModifiedBy_30
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="jcr:lastModifiedBy"/>
<alias
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="sling:alias"/>
<vanityPath
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="sling:vanityPath"/>
<asyncstatus
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="async-status"/>
<asyncstart
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="async-start"/>
<asyncdone
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="async-done"/>
<runtimeId
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="runtimeId"/>
<leaderElectionId
jcr:primaryType="nt:unstructured"
index="{Boolean}false"
name="leaderElectionId"/>
<resourceType
jcr:primaryType="nt:unstructured"
name="sling:resourceType"
propertyIndex="{Boolean}true"
weight="{Long}0"/>
<resourceSuperType
jcr:primaryType="nt:unstructured"
name="sling:resourceSuperType"
propertyIndex="{Boolean}true"
weight="{Long}0"/>
<prop
jcr:primaryType="nt:unstructured"
analyzed="{Boolean}true"
isRegexp="{Boolean}true"
name="^[^\\/]*$"
nodeScopeIndex="{Boolean}true"
useInExcerpt="{Boolean}true"/>
</properties>
</nt:base>
</indexRules>
</lucene>

[arunpatidar · Accepted solution]

configuration look ok but please check if reindexing works.

[mahamooda726898]

I have deleted all indexes and restarted the AEM so that indexes get triggered . I don't see any issues with indexing.

Plus here is the log of query.

GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryEngineImpl Parsing JCR-SQL2 statement: SELECT [rep:suggest()] FROM [cq:Page] WHERE ISDESCENDANTNODE('/content/we-retail/us/en') AND SUGGEST('lady')
26.03.2020 11:35:36.160 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost using filter Filter(query=SELECT [rep:suggest()] FROM [cq:Page] WHERE ISDESCENDANTNODE('/content/we-retail/us/en') AND SUGGEST('lady'), path=/content/we-retail/us/en//*, property=[native*lucene=[suggest?term=lady]])
26.03.2020 11:35:36.160 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for reference is Infinity
26.03.2020 11:35:36.160 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for property is Infinity
26.03.2020 11:35:36.160 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for nodeType is Infinity
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryEngineImpl Parsing xpath statement: /jcr:root/content/we-retail/us/en//element(*, cq:Page)[(jcr:contains(., 'lady'))]
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryEngineImpl XPath > SQL2: select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'lady') and isdescendantnode(a, '/content/we-retail/us/en') /* xpath: /jcr:root/content/we-retail/us/en//element(*, cq:Page)[(jcr:contains(., 'lady'))] */
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost using filter Filter(query=select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'lady') and isdescendantnode(a, '/content/we-retail/us/en') /* xpath: /jcr:root/content/we-retail/us/en//element(*, cq:Page)[(jcr:contains(., 'lady'))] */ fullText="lady", path=/content/we-retail/us/en//*)
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for reference is Infinity
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for property is Infinity
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for nodeType is Infinity
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for [/oak:index/cqPageLucene] of type (lucene-property) with plan [lucene:cqPageLucene(/oak:index/cqPageLucene) *:*] is 2.00
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property[/oak:index/cqPageLucene] is 2.0
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for traverse is Infinity
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query execute SELECT [rep:suggest()] FROM [cq:Page] WHERE ISDESCENDANTNODE('/content/we-retail/us/en') AND SUGGEST('lady')
26.03.2020 11:35:36.161 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-suggestions.suggestions.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query plan [cq:Page] as [cq:Page] /* lucene:cqPageLucene(/oak:index/cqPageLucene) *:* where (isdescendantnode([cq:Page], [/content/we-retail/us/en])) and (suggest([cq:Page], 'lady')) */
26.03.2020 11:35:36.213 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for [/oak:index/lucene] of type (lucene-property) with plan [lucene:lucene(/oak:index/lucene) :fulltext:women :fulltext:womens :fulltext:lady ft:("lady")] is 168352.00
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for [/oak:index/cqPageLucene] of type (lucene-property) with plan [lucene:cqPageLucene(/oak:index/cqPageLucene) +:fulltext:lady +:ancestors:/content/we-retail/us/en ft:("lady")] is 1721.00
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property[/oak:index/cqPageLucene] is 1721.0
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for aggregate lucene is Infinity
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for solr is Infinity
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl cost for traverse is Infinity
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query execute select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'lady') and isdescendantnode(a, '/content/we-retail/us/en') /* xpath: /jcr:root/content/we-retail/us/en//element(*, cq:Page)[(jcr:contains(., 'lady'))] */
26.03.2020 11:35:36.214 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query plan [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene) +:fulltext:lady +:ancestors:/content/we-retail/us/en ft:("lady") where (contains([a].[*], 'lady')) and (isdescendantnode([a], [/content/we-retail/us/en])) */
26.03.2020 11:35:36.216 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query execute select [jcr:path], [jcr:score], * from [cq:Page] as a where contains(*, 'lady') and isdescendantnode(a, '/content/we-retail/us/en') /* xpath: /jcr:root/content/we-retail/us/en//element(*, cq:Page)[(jcr:contains(., 'lady'))] */
26.03.2020 11:35:36.217 *DEBUG* [0:0:0:0:0:0:0:1 [1585202736158] GET /content/summit/l4080/chapter-4/jcr:content/root/responsivegrid/search/quick-results.results.json HTTP/1.1] org.apache.jackrabbit.oak.query.QueryImpl query plan [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene) +:fulltext:lady +:ancestors:/content/we-retail/us/en ft:("lady") where (contains([a].[*], 'lady')) and (isdescendantnode([a], [/content/we-retail/us/en])) */
26.03.2020 11:36:06.480 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryEngineImpl Parsing xpath statement: /jcr:root/var/eventing/jobs//element(*,slingevent:Job)[@event.job.topic = 'org/apache/sling/distribution/queue/publish/endpoint0' and not(@slingevent:finishedState)] order by @slingevent:created ascending
26.03.2020 11:36:06.480 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryEngineImpl XPath > SQL2: select [jcr:path], [jcr:score], * from [slingevent:Job] as a where [event.job.topic] = 'org/apache/sling/distribution/queue/publish/endpoint0' and [slingevent:finishedState] is null and isdescendantnode(a, '/var/eventing/jobs') order by [slingevent:created] /* xpath: /jcr:root/var/eventing/jobs//element(*,slingevent:Job)[@event.job.topic = 'org/apache/sling/distribution/queue/publish/endpoint0' and not(@slingevent:finishedState)] order by @slingevent:created ascending */
26.03.2020 11:36:06.481 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryImpl cost using filter Filter(query=select [jcr:path], [jcr:score], * from [slingevent:Job] as a where [event.job.topic] = 'org/apache/sling/distribution/queue/publish/endpoint0' and [slingevent:finishedState] is null and isdescendantnode(a, '/var/eventing/jobs') order by [slingevent:created] /* xpath: /jcr:root/var/eventing/jobs//element(*,slingevent:Job)[@event.job.topic = 'org/apache/sling/distribution/queue/publish/endpoint0' and not(@slingevent:finishedState)] order by @slingevent:created ascending */, path=/var/eventing/jobs//*, property=[event.job.topic=[org/apache/sling/distribution/queue/publish/endpoint0], slingevent:finishedState=[is null]])
26.03.2020 11:36:06.481 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryImpl cost for reference is Infinity
26.03.2020 11:36:06.481 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryImpl cost for property is Infinity
26.03.2020 11:36:06.481 *DEBUG* [sling-default-4-health-org.apache.sling.distribution.monitor.DistributionQueueHealthCheck] org.apache.jackrabbit.oak.query.QueryImpl cost for nodeType is Infinity

[mahamooda726898]

I have deleted all indexes and restarted the AEM so that indexes get triggered . I don't see any issues with indexing.

[mahamooda726898]

Thanks for your direction , I was doing indexing by deleting the index folder which was wrong. Then I reindex it by using crx/de and functionality started working.

[10024]

Hello good day! I would like to know where should the changes for synonyms be added in the solution. Should it be in the ui.content?

[arunpatidar]

I would add in ui.apps but make sure your POM should have <allowIndexDefinitions>true</allowIndexDefinitions> in filevault-package-maven-plugin plugin configuration

[10024]

Thanks for the response. Will it matter if I add it in ui.content? It's already working on my local but I am encountering some issues once i deploy it to the AEM instance. After code deployment jcr:primaryType gets converted to nt:folder from nt:unstructured. Have you encountered this issue before? 

Related: After code deployment to AEM, jcr:primaryType gets converted to nt:folder from nt:unstructured

[arunpatidar]

No, it will not matter. it is just the way code & content is separated in different modules. You should check the filter.xml and content(.content.xml) for all impacted nodes within this tree for your another issue, it seems you might replacing everything from your package.

[kautuk_sahni]

@Albin_Issac , You wrote a wonderful blog on "Enable search synonyms in AEM"  [1]. Request you to please assist.

[1] https://medium.com/tech-learnings/how-to-enable-search-synonyms-in-aem-with-lucene-ccb780375eb4 

[Albin_Issac]

Thanks @kautuk_sahni

HI @mahamooda726898

Please let me if you still facing the issue after following the blog. Also share the index configuration package

Regards

Albin I

www.albinsblog.com