Issue with an oak index using snonym filter

[lksfjhadfas]

Hey guys,

I have a custom index which is working totally fine if i do not add any analyzers to it. I added an analyzer for adding a synonym to return similar results for "fact sheets" and factsheets. Index stopped working after that.

Any help with it is appreciated.

Thank you!

Here is the index

<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:oak="http://jackrabbit.apache.org/oak/ns/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"

          jcr:primaryType="oak:Unstructured"

          async="async"

          compatVersion="{Long}2"

          evaluatePathRestrictions="{Boolean}true"

          reindex="{Boolean}true"

          type="lucene">

    <indexRules jcr:primaryType="nt:unstructured">

        <nt:base

            jcr:primaryType="nt:unstructured"

            includePropertyTypes="all">

            <properties jcr:primaryType="nt:unstructured">

                <literatureTitle name="literatureTitle"

                    analyzed="{Boolean}true"

                    ordered="{Boolean}true"

                    jcr:primaryType="nt:unstructured"/>

                <displayContentTypename name="displayContentTypename"

                    analyzed="{Boolean}true"

                    jcr:primaryType="nt:unstructured"/>

            </properties>

        </nt:base>

    </indexRules>

    <analyzers jcr:primaryType="nt:unstructured">

        <default jcr:primaryType="nt:unstructured">

            <filters jcr:primaryType="nt:unstructured">

                <LowerCase jcr:primaryType="nt:unstructured"/>

                <Synonym jcr:primaryType="nt:unstructured"

                synonyms="synonym.txt">

                    <synonym.txt/>

                </Synonym>

            </filters>

        </default>

    </analyzers>

</jcr:root>

and in synonym.txt i have

fact sheets, factsheets

[smacdonald2008]

Have you read somewhere that this is supported? I am checking internally.

[lksfjhadfas]

i created that using this as an example. AEM Search Indexing: Synonyms, Filters, and Stop Words (oh my!) | HS2 Solutions

Is there any way i cann achieve this where factsheet and fact sheets returns the same results.

[smacdonald2008]

Thanks for the information. In your example - have you followed all the information given here - Understanding Analyzers, Tokenizers, and Filters | Apache Solr Reference Guide 6.6

[lksfjhadfas]

we are not using Solr index. just the lucene full text index. And the index is in the same format as the example. But, i am not sure why its not working

Thanks!

[Jörg_Hoh]

What do you mean with "it stopped working"? Any exceptions? What change in behaviour did you find when you changed the index definition?

[lksfjhadfas]

when i try to reindex it, it just stays true and doesnt change to false and i dont see anything in the logs either.

[Jörg_Hoh]

Ok, so if you add these additional settings and you try to reindex, the reindexing is not starting? Just from looking at the index definition I would assume that the nodetype of the index definition itself (/oak:index/customIndex) is wrong; it should not be "nt:unstructured" but rather "oak:queryIndexDefinition".

Jörg

[lksfjhadfas]

If i use the oak:queryIndexDefinition, it gives me a "javax.jcr.nodetype.ConstraintViolationException: OakConstraint0001: The primary type null does not exist (500)" error. I am not sure what the issue is.

[lksfjhadfas]

Nvm i had to delete the existing index for it to take the new node type. I will try adding the synonym now and see if it works and let you know.

Thanks!

[lksfjhadfas]

After i changed the node type, i had to add a tokenizer for the index to be reindexes.

<analyzers jcr:primaryType="nt:unstructured">

        <default

            jcr:primaryType="nt:unstructured">

            <filters jcr:primaryType="nt:unstructured">

                <Synonym jcr:primaryType="nt:unstructured"

                    synonyms="synonym.txt">

                    <synonym.txt/>

                </Synonym>

            </filters>

            <tokenizer jcr:primaryType="nt:unstructured"

                       name="classic"/>

        </default>

    </analyzers>

now i can re index but i am not getting any results back using this index.

[Jörg_Hoh]

Can you please provide your complete index definition (e.g. as JSON dump)? It's hard to guess just from this snippet what could be wrong.

[lksfjhadfas]

Here is the JSON of the index

"faiLiteratureIndex":{

   "jcr:primaryType":"oak:QueryIndexDefinition",

   "compatVersion":2,

   "type":"lucene",

   "async":"async",

   "evaluatePathRestrictions":true,

   "reindex":true,

   "indexRules":{

      "jcr:primaryType":"nt:unstructured",

      "nt:base":{

         "jcr:primaryType":"nt:unstructured",

         "includePropertyTypes":"all",

         "properties":{

            "jcr:primaryType":"nt:unstructured",

            "literatureTitle":{

               "jcr:primaryType":"nt:unstructured",

               "ordered":true,

               "analyzed":true,

               "name":"literatureTitle"

            },

            "displayContentTypename":{

               "jcr:primaryType":"nt:unstructured",

               "analyzed":true,

               "name":"displayContentTypename"

            }

         }

      }

   },

   "analyzers":{

      "jcr:primaryType":"nt:unstructured",

      "default":{

         "jcr:primaryType":"nt:unstructured",

         "filters":{

            "jcr:primaryType":"nt:unstructured",

            "Synonym":{

               "jcr:primaryType":"nt:unstructured",

               "synonyms":"synonym.txt"

            }

         },

         "tokenizer":{

            "jcr:primaryType":"nt:unstructured",

            "name":"classic"

         }

      }

   }

}

[Jörg_Hoh · Accepted solution]

As I read the docs [1], you should have the synonyms.txt as nt:file in the "Synonym" node; and is there a "classicTokenizerFactory"? I would try with "name" :"Classic" (uppercase).

Jörg

[1] Jackrabbit Oak – Lucene Index

[lksfjhadfas]

I am able to get the results back. I also had to add lowercase filter for it to work. Thank you so much Jörg Hoh​.