AEM 6.2 - Oak Lucene Indexes - Configuring Composing Analyzer

[maximen66097493]

Hello,

I'm looking for a way to have a configuration using Composing Analyzer in order to have a different filters between the analyzer used for the index and the query.

On the Oak Lucene Documentation ( Jackrabbit Oak – Lucene Index ​) i see that we can configure default or pathText analyzers and maybe others due to the ... but i can't find the documentation related to the exhaustive list of the analyzers we can use.

Oak Lucene documentation

+ sampleIndex
   - jcr:primaryType = "oak:QueryIndexDefinition"
   + analyzers
     + default
     + pathText
     ...

They seems also to be different than the one in the Apache Solr documentation ( Analyzers - Apache Solr Reference Guide - Apache Software Foundation  ): index and query

<analyzer type="index">

    <tokenizer class="solr.StandardTokenizerFactory"/>

    <filter class="solr.LowerCaseFilterFactory"/>

    <filter class="solr.KeepWordFilterFactory" words="keepwords.txt"/>

    <filter class="solr.SynonymFilterFactory" synonyms="syns.txt"/>

</analyzer>

<analyzer type="query">

    <tokenizer class="solr.StandardTokenizerFactory"/>

    <filter class="solr.LowerCaseFilterFactory"/>

</analyzer>

Does anyone know where i could find the documentation on the list of configurable analyzers for Oak Lucene index or how can i configure a different analyzer for query and for index ?

Best regards,

Maxime Nougarede

DigitasLbi

[smacdonald2008]

I have asked within Adobe if anyone knows if such docs exist.

[maximen66097493]

Hi,

Thanks a lot for your help, meanwhile i found a part of my answer in the Jackrabbit Oak – Lucene Index documentation where they wrote:

@Note that currently only one analyzer can be configured per index. Its not possible to specify separate analyzer for query and index time currently.

https://jackrabbit.apache.org/oak/docs/query/lucene.html

But i still want to understand difference between default and pathText .

[Peter_Puzanovs]

Dear Maxim,

Thank you for asking such interesting question indeed,

Looking at:

org/apache/jackrabbit/oak/plugins/index/lucene/IndexDefinition.java

We can see that the assembly is happening in the:

private static Map<String, Analyzer> collectAnalyzers(NodeState defn) {

  Map<String, Analyzer> analyzerMap = newHashMap();

  NodeStateAnalyzerFactory factory = new NodeStateAnalyzerFactory(LuceneIndexConstants.VERSION);

  NodeState analyzersTree = defn.getChildNode(LuceneIndexConstants.ANALYZERS);

   for (ChildNodeEntry cne : analyzersTree.getChildNodeEntries()) {

  Analyzer a = factory.createInstance(cne.getNodeState());

  analyzerMap.put(cne.getName(), a);

  }

   if (getOptionalValue(analyzersTree, INDEX_ORIGINAL_TERM, false) && !analyzerMap.containsKey(ANL_DEFAULT)) {

  analyzerMap.put(ANL_DEFAULT, new OakAnalyzer(VERSION, true));

  }

   return ImmutableMap.copyOf(analyzerMap);

}

Which then are set during the IndexDefinition build to variable:

this.analyzers = collectAnalyzers(defn);

Which then are used in the following area:

if (analyzers.containsKey(LuceneIndexConstants.ANL_DEFAULT)){

  defaultAnalyzer = analyzers.get(LuceneIndexConstants.ANL_DEFAULT);

}

So, to answer your question. You can define as many analysers as you want, but as per current code base only the 'default' analyser will be used.

Regards,

Peter

[ThomasMueller1]

We found this incorrect, and will remove "pathText" from the documentation. Sorry for the delay!