I am trying to use Java High Level Rest Client in Adobe Experience Manager to finish project of comparisson between Lucene, Solr and Elasticsearch search engines.I am having some problems with dependencies that I don't understand so any info on this would be valid.Here is the code:Dependencies in the parent pom.xml (the same are defined in core pom.xml):<dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>7.4.0</version></dependency><dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-client</artifactId> <version>7.4.0</version></dependency><dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.4.0</version></dependency>The only line of code that I am using that is from dependencies abovetry (RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(server, port, protocol), new HttpHost(server, secondPort, protocol)));){}catch (ElasticsearchException e){ LOG.error("Exception: " + e);}protocol = "http", server = "localhost", port = 9200, secondPort = 9201ErrorDependencies from IntelliJI know that there is usually problem with dependencies versions, but all are 7.4.0 in this case. Also elasticsearch 7.4.0v is running locally on 3 nodes.This project is done on We.Retail project so it is easy to replicate. Also all the code with this error is available here: https://github.com/tadijam64/search-engines-comparison-on-we-retail/tree/elasticsearch-integrationAny info or idea is appreciated.

Level 3

Solved

Using Elasticsearch Rest High Client problem in AEM

Forum|Forum|6 years ago
October 18, 2019
6 replies
5991 views

I am trying to use Java High Level Rest Client in Adobe Experience Manager to finish project of comparisson between Lucene, Solr and Elasticsearch search engines.

I am having some problems with dependencies that I don't understand so any info on this would be valid.
Here is the code:

Dependencies in the parent pom.xml (the same are defined in core pom.xml):

<groupId>org.elasticsearch</groupId>

<artifactId>elasticsearch</artifactId>

</dependency>

<groupId>org.elasticsearch.client</groupId>

<artifactId>elasticsearch-rest-client</artifactId>

</dependency>

<groupId>org.elasticsearch.client</groupId>

<artifactId>elasticsearch-rest-high-level-client</artifactId>

</dependency>

The only line of code that I am using that is from dependencies above

try (RestHighLevelClient client = new

RestHighLevelClient(RestClient.builder(new HttpHost(server, port,

protocol),

new HttpHost(server, secondPort, protocol)));)

{

}

catch (ElasticsearchException e)

{

LOG.error("Exception: " + e);

}

protocol = "http", server = "localhost", port = 9200, secondPort = 9201

Error

Dependencies from IntelliJ

I know that there is usually problem with dependencies versions, but all are 7.4.0 in this case. Also elasticsearch 7.4.0v is running locally on 3 nodes.

This project is done on We.Retail project so it is easy to replicate. Also all the code with this error is available here: https://github.com/tadijam64/search-engines-comparison-on-we-retail/tree/elasticsearch-integration

Any info or idea is appreciated.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by tadijam47062732

So my guess was right, transitive dependencies were not included altho Transitive>true</Embed-Transitive> exists.

The following is necessary when running elasticsearch as search engine on AEM the problem:

<!-- Elasticsearch -->
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-high-level-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch-x-content</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>rank-eval-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-imaging</artifactId>
       <scope>test</scope>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>lang-mustache-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.httpcomponents</groupId>
       <artifactId>httpasyncclient</artifactId>
  </dependency>

It is important to add all third-party dependencies as <Embed-Dependency> inside maven-bundle-plugin like this:

<plugin>
   <groupId>org.apache.felix</groupId>
   <artifactId>maven-bundle-plugin</artifactId>
   <extensions>true</extensions>
   <configuration>
        <instructions>
             <Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, noggit,
                 elasticsearch-rest-high-level-client,
                 elasticsearch,
                 elasticsearch-rest-client,
                 elasticsearch-x-content,
                 elasticsearch-core,
                 rank-eval-client,
                 lang-mustache-client,
                 httpasyncclient;
             </Embed-Dependency>
             <Embed-Transitive>true</Embed-Transitive>
             <Embed-Directory>OSGI-INF/lib</Embed-Directory>
             <Export-Package>we.retail.core.model*</Export-Package>
             <Import-Package>
                 *;resolution:=optional
             </Import-Package>
             <Private-Package>
                 we.retail.core*
             </Private-Package>
             <Sling-Model-Packages>
                 we.retail.core.model
             </Sling-Model-Packages>
             <_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>
        </instructions>
   </configuration>
  </plugin>

Important to notice:
All third-party dependencies (the ones outside of OSGi) must be included in the
must be set to true to include transitive dependencies
must include "*;resolution:=optional" to exclude all dependencies that could not be resolved so that program can run normally
For some reason, there was an error in compile time when "elasticsearch" dependency was added which is not important for this task, so I've decided to ignore it this way:

<_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>

Though challenge, but I finally resolved it. There are many similar or the same problems on Google, so I hope this will help someone. Thanks to everyone that tried to help.

Ankur_Khare

Community Advisor

Few stuff to look for-

1. is your bundle active?

2. If it's not active could you please check the dependencies missing?

3. If they are elastic search could you please install elastic search bundles directly to system/console

4. But we should be directly embedding the dependencies directly with the bundle for this we should be adding the external dependencies under this tag in ur core pom.xml-

<groupId>org.apache.felix</groupId>

<artifactId>maven-bundle-plugin</artifactId>

<!-- Import any version of javax.inject, to allow running on multiple

versions of AEM -->

<Import-Package>org.elasticsearch;version=7.4.0,*</Import-Package>

</instructions>

</configuration>

</plugin>

tadijam47062732Author

Level 3

If I do it this way:

<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<executions>
<execution>
<id>bundle-manifest</id>
<phase>process-classes</phase>
<goals>
<goal>manifest</goal>
</goals>
</execution>
</executions>
<configuration>
<instructions>

<Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, log4j, noggit, zookeeper, elasticsearch-rest-client,

elasticsearch-rest-high-level-client, org.apache.servicemix.bundles.elasticsearch, elasticsearch; inline=true

</Embed-Dependency>
<Embed-Transitive>false</Embed-Transitive>
<Embed-Directory>OSGI-INF/lib</Embed-Directory>
<Export-Package>we.retail.core.model*</Export-Package>
<Import-Package>
*;resolution:=optional, org.elasticsearch;version=7.4.0,*

</Import-Package>
<Private-Package>we.retail.core*</Private-Package>
<Sling-Model-Packages>
we.retail.core.model

</Sling-Model-Packages>
</instructions>
</configuration>
</plugin>

Then I am not able to build the project because of the following error:

[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:3.3.0:manifest (bundle-manifest) on project we.retail.core: Error(s) found in manifest configuration -> [Help 1]

This is building correctly if I remove "elasticsearch" artifactId from Embed-Dependency, but still does not work because there are many (7-8) unresolved bundels in there.

Ankur_Khare

Community Advisor

Please use this just made it work in my local instance-

<groupId>org.apache.felix</groupId>

<artifactId>maven-bundle-plugin</artifactId>

<Embed-Dependency>elasticsearch,elasticsearch-rest-client,elasticsearch-rest-high-level-client</Embed-Dependency>

<Embed-Transitive>true</Embed-Transitive>

<Import-Package>!com.carrotsearch.*,

!com.sun.jna.*,

!joptsimple.*,

!org.HdrHistogram.*,

!org.apache.logging.*,

!org.apache.lucene.*,

!org.elasticsearch.common.*,

!org.elasticsearch.core.*,

!org.elasticsearch.geometry.*,

!org.elasticsearch.index.*,

!org.elasticsearch.script.*,

!org.elasticsearch.secure.*,

!org.locationtech.*,

!sun.misc.*,

!com.sun.management.*,

!com.tdunning.math.stats.*,

!org.apache.http.*,

!org.elasticsearch.secure_sm,

*

</Import-Package>

</instructions>

</configuration>

</plugin>

tadijam47062732Author

Level 3

Thank you for the effort, but it does not work in my case. Forbidding importing of packages that I am going to use should not fix the issue.

I can not add an "elasticsearch" artifactId in <Embed-Dependency> because then build fails.

Can you please try to clone my repo and try to make your changes, because I had no success in doing so?

GitHub - tadijam64/search-engines-comparison-on-we-retail at elasticsearch_servlet_impl

The idea is to access some page (for example http://localhost:4902/editor.html/content/we-retail/language-masters/en/products.html ) and try to index pages with elasticsearch option and not to get a NoClassDefFoundError.

tadijam47062732AuthorAccepted solution

Level 3

So my guess was right, transitive dependencies were not included altho Transitive>true</Embed-Transitive> exists.

The following is necessary when running elasticsearch as search engine on AEM the problem:

<!-- Elasticsearch -->
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-high-level-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch-x-content</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>rank-eval-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-imaging</artifactId>
       <scope>test</scope>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>lang-mustache-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.httpcomponents</groupId>
       <artifactId>httpasyncclient</artifactId>
  </dependency>

It is important to add all third-party dependencies as <Embed-Dependency> inside maven-bundle-plugin like this:

<plugin>
   <groupId>org.apache.felix</groupId>
   <artifactId>maven-bundle-plugin</artifactId>
   <extensions>true</extensions>
   <configuration>
        <instructions>
             <Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, noggit,
                 elasticsearch-rest-high-level-client,
                 elasticsearch,
                 elasticsearch-rest-client,
                 elasticsearch-x-content,
                 elasticsearch-core,
                 rank-eval-client,
                 lang-mustache-client,
                 httpasyncclient;
             </Embed-Dependency>
             <Embed-Transitive>true</Embed-Transitive>
             <Embed-Directory>OSGI-INF/lib</Embed-Directory>
             <Export-Package>we.retail.core.model*</Export-Package>
             <Import-Package>
                 *;resolution:=optional
             </Import-Package>
             <Private-Package>
                 we.retail.core*
             </Private-Package>
             <Sling-Model-Packages>
                 we.retail.core.model
             </Sling-Model-Packages>
             <_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>
        </instructions>
   </configuration>
  </plugin>

Important to notice:
All third-party dependencies (the ones outside of OSGi) must be included in the
must be set to true to include transitive dependencies
must include "*;resolution:=optional" to exclude all dependencies that could not be resolved so that program can run normally
For some reason, there was an error in compile time when "elasticsearch" dependency was added which is not important for this task, so I've decided to ignore it this way:

<_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>

Though challenge, but I finally resolved it. There are many similar or the same problems on Google, so I hope this will help someone. Thanks to everyone that tried to help.

H

harishm58550198

The git location is invalid. Can you please update?

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded