Using Elasticsearch Rest High Client problem in AEM

[tadijam47062732]

I am trying to use Java High Level Rest Client in Adobe Experience Manager to finish project of comparisson between Lucene, Solr and Elasticsearch search engines.

I am having some problems with dependencies that I don't understand so any info on this would be valid.
Here is the code:

• Dependencies in the parent pom.xml (the same are defined in core pom.xml):

<!-- Elasticseach dependencies -->

<dependency>

    <groupId>org.elasticsearch</groupId>

    <artifactId>elasticsearch</artifactId>

    <version>7.4.0</version>

</dependency>

<dependency>

    <groupId>org.elasticsearch.client</groupId>

    <artifactId>elasticsearch-rest-client</artifactId>

    <version>7.4.0</version>

</dependency>

<dependency>

    <groupId>org.elasticsearch.client</groupId>

    <artifactId>elasticsearch-rest-high-level-client</artifactId>

    <version>7.4.0</version>

</dependency>

• The only line of code that I am using that is from dependencies above

try (RestHighLevelClient client = new

RestHighLevelClient(RestClient.builder(new HttpHost(server, port,

protocol),

  new HttpHost(server, secondPort, protocol)));)

{

}

catch (ElasticsearchException e)

{

    LOG.error("Exception: " + e);

}

protocol = "http", server = "localhost", port = 9200, secondPort = 9201

• Error

• Dependencies from IntelliJ

I know that there is usually problem with dependencies versions, but all are 7.4.0 in this case. Also elasticsearch 7.4.0v is running locally on 3 nodes.

This project is done on We.Retail project so it is easy to replicate. Also all the code with this error is available here: https://github.com/tadijam64/search-engines-comparison-on-we-retail/tree/elasticsearch-integration

Any info or idea is appreciated.

[Ankur_Khare]

Few stuff to look for-

1. is your bundle active?

2. If it's not active could you please check the dependencies missing?

3. If they are elastic search could you please install elastic search bundles directly to system/console

4. But we should be directly embedding the dependencies directly with the bundle for this we should be adding the external dependencies under this tag in ur core pom.xml-

<plugin>

<groupId>org.apache.felix</groupId>

<artifactId>maven-bundle-plugin</artifactId>

<extensions>true</extensions>

<configuration>

<instructions>

<!-- <Embed-Dependency> artifactId1, artifactId2;inline=true </Embed-Dependency> -->

<!-- Import any version of javax.inject, to allow running on multiple

versions of AEM -->

  <Import-Package>org.elasticsearch;version=7.4.0,*</Import-Package>

</instructions>

</configuration>

</plugin>

[tadijam47062732]

If I do it this way:

<plugin>
  <groupId>org.apache.felix</groupId>
  <artifactId>maven-bundle-plugin</artifactId>
  <extensions>true</extensions>
  <executions>
  <execution>
  <id>bundle-manifest</id>
  <phase>process-classes</phase>
  <goals>
  <goal>manifest</goal>
  </goals>
  </execution>
  </executions>
  <configuration>
  <instructions>
   <!-- <Embed-Dependency> artifactId1, artifactId2;inline=true </Embed-Dependency> -->
   <Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, log4j, noggit, zookeeper, elasticsearch-rest-client,

  elasticsearch-rest-high-level-client, org.apache.servicemix.bundles.elasticsearch, elasticsearch; inline=true

   </Embed-Dependency>
  <Embed-Transitive>false</Embed-Transitive>
  <Embed-Directory>OSGI-INF/lib</Embed-Directory>
  <Export-Package>we.retail.core.model*</Export-Package>
  <Import-Package>
   *;resolution:=optional, org.elasticsearch;version=7.4.0,*

   </Import-Package>
  <Private-Package>we.retail.core*</Private-Package>
  <Sling-Model-Packages>
   we.retail.core.model

   </Sling-Model-Packages>
  </instructions>
  </configuration>
</plugin>

Then I am not able to build the project because of the following error:

[ERROR] Failed to execute goal org.apache.felix:maven-bundle-plugin:3.3.0:manifest (bundle-manifest) on project we.retail.core: Error(s) found in manifest configuration -> [Help 1]

This is building correctly if I remove "elasticsearch" artifactId from Embed-Dependency, but still does not work because there are many (7-8) unresolved bundels in there.

[Ankur_Khare]

Please use this just made it work in my local instance-

<plugin>

<groupId>org.apache.felix</groupId>

<artifactId>maven-bundle-plugin</artifactId>

<extensions>true</extensions>

<configuration>

<instructions>

<Embed-Dependency>elasticsearch,elasticsearch-rest-client,elasticsearch-rest-high-level-client</Embed-Dependency>

            <Embed-Transitive>true</Embed-Transitive>

            <Import-Package>!com.carrotsearch.*,

            !com.sun.jna.*,

            !joptsimple.*,

            !org.HdrHistogram.*,

            !org.apache.logging.*,

            !org.apache.lucene.*,

            !org.elasticsearch.common.*,

            !org.elasticsearch.core.*,

            !org.elasticsearch.geometry.*,

            !org.elasticsearch.index.*,

            !org.elasticsearch.script.*,

            !org.elasticsearch.secure.*,

            !org.locationtech.*,

            !sun.misc.*,

            !com.sun.management.*,

            !com.tdunning.math.stats.*,

            !org.apache.http.*,

            !org.elasticsearch.secure_sm,

            *

            </Import-Package>

</instructions>

</configuration>

</plugin>

[tadijam47062732]

Thank you for the effort, but it does not work in my case. Forbidding importing of packages that I am going to use should not fix the issue.

I can not add an "elasticsearch" artifactId in <Embed-Dependency> because then build fails.

Can you please try to clone my repo and try to make your changes, because I had no success in doing so?

GitHub - tadijam64/search-engines-comparison-on-we-retail at elasticsearch_servlet_impl

The idea is to access some page (for example http://localhost:4902/editor.html/content/we-retail/language-masters/en/products.html ) and try to index pages with elasticsearch option and not to get a NoClassDefFoundError.

[tadijam47062732 · Accepted solution]

So my guess was right, transitive dependencies were not included altho Transitive>true</Embed-Transitive> exists.

The following is necessary when running elasticsearch as search engine on AEM the problem:

<!-- Elasticsearch -->
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-high-level-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.client</groupId>
       <artifactId>elasticsearch-rest-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch</groupId>
       <artifactId>elasticsearch-x-content</artifactId>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>rank-eval-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-imaging</artifactId>
       <scope>test</scope>
  </dependency>
  <dependency>
       <groupId>org.elasticsearch.plugin</groupId>
       <artifactId>lang-mustache-client</artifactId>
  </dependency>
  <dependency>
       <groupId>org.apache.httpcomponents</groupId>
       <artifactId>httpasyncclient</artifactId>
  </dependency>

It is important to add all third-party dependencies as <Embed-Dependency> inside maven-bundle-plugin like this:

<plugin>
   <groupId>org.apache.felix</groupId>
   <artifactId>maven-bundle-plugin</artifactId>
   <extensions>true</extensions>
   <configuration>
        <instructions>
             <Embed-Dependency>org.apache.servicemix.bundles.solr-solrj, noggit,
                 elasticsearch-rest-high-level-client,
                 elasticsearch,
                 elasticsearch-rest-client,
                 elasticsearch-x-content,
                 elasticsearch-core,
                 rank-eval-client,
                 lang-mustache-client,
                 httpasyncclient;
             </Embed-Dependency>
             <Embed-Transitive>true</Embed-Transitive>
             <Embed-Directory>OSGI-INF/lib</Embed-Directory>
             <Export-Package>we.retail.core.model*</Export-Package>
             <Import-Package>
                 *;resolution:=optional
             </Import-Package>
             <Private-Package>
                 we.retail.core*
             </Private-Package>
             <Sling-Model-Packages>
                 we.retail.core.model
             </Sling-Model-Packages>
             <_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>
        </instructions>
   </configuration>
  </plugin>

Important to notice:

• All third-party dependencies (the ones outside of OSGi) must be included in the
• must be set to true to include transitive dependencies
• must include "*;resolution:=optional" to exclude all dependencies that could not be resolved so that program can run normally
• For some reason, there was an error in compile time when "elasticsearch" dependency was added which is not important for this task, so I've decided to ignore it this way:

<_fixupmessages>"Classes found in the wrong directory";is:=warning</_fixupmessages>

Though challenge, but I finally resolved it. There are many similar or the same problems on Google, so I hope this will help someone. Thanks to everyone that tried to help.

[harishm58550198]

The git location is invalid. Can you please update?