Apache Kafka Integration with AEM | Community
Skip to main content
N
Navneet_Saxena-1
July 21, 2021
Solved

Apache Kafka Integration with AEM

  • July 21, 2021
  • 3 replies
  • 4378 views

Hi All,

We are trying to integrate AEM with Apache Kafka and are trying to use Apache Kafka Jars (i.e Kafka-clients & Kafka-avro-serializer) in order to integrate. We are able to build the code and deploy.

But on OSGI we are not able to resolve the dependency as the Apache Kafka Jars are not bundles. We have tried putting in the Import Package and Embed Dependency as well but nothing worked.

Can anyone please help on the same?

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by joerghoh

Apache Service-Mix provides OSGI-ified kafka packages.

 

I integrated it some years back, the code is available at https://github.com/joerghoh/kafka-integration. You probably should update the versions 🙂

 

Jörg

3 replies

Vijayalakshmi_S
Vijayalakshmi_S
July 22, 2021

Hi @navneet_saxena-1,

If it is allowed to be installed as separate OSGi bundle outside your project code base, you can make use of simple maven project with maven-bundle-plugin and desired dependency. 

When you try in your local for the first time, you can try with Export-Package -> *. 

Once when you see the bundle being active and functionality works without NoClassDefFound Error, you can explicitly mention the desired packages in Export-Package entry as opposed to "*" -> test the functionality again.

https://myaemlearnings.blogspot.com/2021/02/createbuild-and-install-osgi-bundle-of.html

 

 

    A
    AEMnewbie
    June 6, 2022

    Hello @vijayalakshmi_s I tried following the above steps ..but i still get the ClassNotFoundException. 

     

    I added the required class package i needed in here 

     

    <Import-Package>

        org.apache.kafka.common.security.plain,

    </Import-Package>

    <Export-Package>

        org.apache.kafka.common.security.plain.*;version="3.0.1",

    </Export-Package> Is there any thing else that we could try to get aaround it?

    joerghoh
    Adobe Employee
    joerghohAdobe EmployeeAccepted solution
    Adobe Employee
    July 22, 2021

    Apache Service-Mix provides OSGI-ified kafka packages.

     

    I integrated it some years back, the code is available at https://github.com/joerghoh/kafka-integration. You probably should update the versions 🙂

     

    Jörg

      C
      chakravarthysvsn
      March 8, 2022

      Hi I am also having same problem.. did you solve this issue
      Please help me if you already got the solution.

      HeenaMadan
      Community Advisor and Adobe Champion
      HeenaMadanCommunity Advisor and Adobe Champion
      Community Advisor and Adobe Champion
      July 11, 2022

      apache servicemix provides dependency. use below in main(root) pom's dependencymanagement

      <dependency>
         <groupId>org.apache.servicemix.bundles</groupId>
         <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
         <version>3.0.0_1</version>
      </dependency>

      same in core pim.xml

      <dependency>
          <groupId>org.apache.servicemix.bundles</groupId>
          <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
      </dependency>

      embedded these dependencies in depdencies or all pom.xml(according to your project structure)

      <embedded>
          <groupId>org.apache.servicemix.bundles</groupId>
          <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
          <target>/apps/<project-package>/application/install</target>
      </embedded>

       

      Terms & ConditionsAccessibility statement

      Loading footer...