Apache Kafka Integration with AEM

[Navneet_Saxena]

Hi All,

We are trying to integrate AEM with Apache Kafka and are trying to use Apache Kafka Jars (i.e Kafka-clients & Kafka-avro-serializer) in order to integrate. We are able to build the code and deploy.

But on OSGI we are not able to resolve the dependency as the Apache Kafka Jars are not bundles. We have tried putting in the Import Package and Embed Dependency as well but nothing worked.

Can anyone please help on the same?

[Vijayalakshmi_S]

Hi @Navneet_Saxena,

If it is allowed to be installed as separate OSGi bundle outside your project code base, you can make use of simple maven project with maven-bundle-plugin and desired dependency. 

When you try in your local for the first time, you can try with Export-Package -> *. 

Once when you see the bundle being active and functionality works without NoClassDefFound Error, you can explicitly mention the desired packages in Export-Package entry as opposed to "*" -> test the functionality again.

https://myaemlearnings.blogspot.com/2021/02/createbuild-and-install-osgi-bundle-of.html

[AEMnewbie]

Hello @Vijayalakshmi_S I tried following the above steps ..but i still get the ClassNotFoundException. 

I added the required class package i needed in here 

<Import-Package>

    org.apache.kafka.common.security.plain,

</Import-Package>

<Export-Package>

    org.apache.kafka.common.security.plain.*;version="3.0.1",

</Export-Package> Is there any thing else that we could try to get aaround it?

[Jörg_Hoh · Accepted solution]

Apache Service-Mix provides OSGI-ified kafka packages.

I integrated it some years back, the code is available at https://github.com/joerghoh/kafka-integration. You probably should update the versions

Jörg

[chakravarthysvsn]

Hi I am also having same problem.. did you solve this issue
Please help me if you already got the solution.

[HeenaMadan]

apache servicemix provides dependency. use below in main(root) pom's dependencymanagement

<dependency>
   <groupId>org.apache.servicemix.bundles</groupId>
   <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
   <version>3.0.0_1</version>
</dependency>

same in core pim.xml

<dependency>
    <groupId>org.apache.servicemix.bundles</groupId>
    <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
</dependency>

embedded these dependencies in depdencies or all pom.xml(according to your project structure)

<embedded>
    <groupId>org.apache.servicemix.bundles</groupId>
    <artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
    <target>/apps/<project-package>/application/install</target>
</embedded>