Lucene search for Chinese market | Community
Skip to main content
V
Vazahat_Fatima_
Level 2
October 26, 2018
Solved

Lucene search for Chinese market

  • October 26, 2018
  • 1 reply
  • 1572 views

In our application, we have multiple market search, both pages and dam. Below is our basic query. Search works fine for most markets, but for China market, seems like we are getting lot of irrelevant results. Not able to find how Lucene search is working for non-English languages. Are there any configurations required for this? Also, do we have any specific Analyzer configuration for Chinese language?

fulltext=insurance

1_group.p.or=true

1_group.1_group.path=/content/product/us

1_group.2_group.path=/content/dam/product/us

2_group.p.or=true

2_group.1_group.type=cq:Page

2_group.2_group.type=dam:Asset

p.excerpt=true

AEM version: 6.2.0, SP1 - CFP8

Oak version: Apache Jackrabbit Oak 1.4.17

Thanks,

Vazahat Fatima P

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Peter_Puzanovs

Dear Vazahat,

Normally, AEM tries to index for English Language, Lucene by standard also has everything configured for english language, indexes are also setup to follow English semantics.

AEM/OAK/Lucene/Java does not do any magic, it only crunches your data into numbers(hashes/hello inverted index), compares numbers of the matches and shows you them in the certain order. When you get irrelevant results it means that your indexes cotain irrelevant data. Therefore you need to correct:

a) How the data get's into your indexes

b) How you retrieve data from your indexes

It's fairly hard to get this 'right' just with plain Oak-Lucene integration.[0]

Please consider using Oak Solr extension[1] that provide support for Chinese language and human readable format of configuration.

Also, can recommend recent book on Relevancy by Doug[2]

[0] Issue with an oak index using snonym filter

[1] Language Analysis | Apache Solr Reference Guide 6.6

[2] Relevant Search: With applications for Solr and Elasticsearch: Doug Turnbull, John Berryman: 9781617292774: Amazon.com: …

Regards,

Peter

1 reply

Peter_Puzanovs
Community Advisor
Peter_PuzanovsCommunity AdvisorAccepted solution
Community Advisor
October 26, 2018

Dear Vazahat,

Normally, AEM tries to index for English Language, Lucene by standard also has everything configured for english language, indexes are also setup to follow English semantics.

AEM/OAK/Lucene/Java does not do any magic, it only crunches your data into numbers(hashes/hello inverted index), compares numbers of the matches and shows you them in the certain order. When you get irrelevant results it means that your indexes cotain irrelevant data. Therefore you need to correct:

a) How the data get's into your indexes

b) How you retrieve data from your indexes

It's fairly hard to get this 'right' just with plain Oak-Lucene integration.[0]

Please consider using Oak Solr extension[1] that provide support for Chinese language and human readable format of configuration.

Also, can recommend recent book on Relevancy by Doug[2]

[0] Issue with an oak index using snonym filter

[1] Language Analysis | Apache Solr Reference Guide 6.6

[2] Relevant Search: With applications for Solr and Elasticsearch: Doug Turnbull, John Berryman: 9781617292774: Amazon.com: …

Regards,

Peter

Terms & ConditionsAccessibility statement

Loading footer...