Expand my Community achievements bar.

Dive into Adobe Summit 2024! Explore curated list of AEM sessions & labs, register, connect with experts, ask questions, engage, and share insights. Don't miss the excitement.
SOLVED

Apache Kafka Integration with AEM

Avatar

Level 1

Hi All,

We are trying to integrate AEM with Apache Kafka and are trying to use Apache Kafka Jars (i.e Kafka-clients & Kafka-avro-serializer) in order to integrate. We are able to build the code and deploy.

But on OSGI we are not able to resolve the dependency as the Apache Kafka Jars are not bundles. We have tried putting in the Import Package and Embed Dependency as well but nothing worked.

Can anyone please help on the same?

1 Accepted Solution

Avatar

Correct answer by
Employee Advisor

Apache Service-Mix provides OSGI-ified kafka packages.

 

I integrated it some years back, the code is available at https://github.com/joerghoh/kafka-integration. You probably should update the versions

 

Jörg

View solution in original post

5 Replies

Avatar

Community Advisor

Hi @Navneet_Saxena,

If it is allowed to be installed as separate OSGi bundle outside your project code base, you can make use of simple maven project with maven-bundle-plugin and desired dependency. 

When you try in your local for the first time, you can try with Export-Package -> *. 

Once when you see the bundle being active and functionality works without NoClassDefFound Error, you can explicitly mention the desired packages in Export-Package entry as opposed to "*" -> test the functionality again.

https://myaemlearnings.blogspot.com/2021/02/createbuild-and-install-osgi-bundle-of.html

 

 

Avatar

Level 4

Hello @Vijayalakshmi_S I tried following the above steps ..but i still get the ClassNotFoundException. 

 

I added the required class package i needed in here 

 

<Import-Package>

    org.apache.kafka.common.security.plain,

</Import-Package>

<Export-Package>

    org.apache.kafka.common.security.plain.*;version="3.0.1",

</Export-Package> Is there any thing else that we could try to get aaround it?

Avatar

Correct answer by
Employee Advisor

Apache Service-Mix provides OSGI-ified kafka packages.

 

I integrated it some years back, the code is available at https://github.com/joerghoh/kafka-integration. You probably should update the versions

 

Jörg

Avatar

Level 1

Hi I am also having same problem.. did you solve this issue
Please help me if you already got the solution.

Avatar

Community Advisor

apache servicemix provides dependency. use below in main(root) pom's dependencymanagement

<dependency>
<groupId>org.apache.servicemix.bundles</groupId>
<artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
<version>3.0.0_1</version>
</dependency>

same in core pim.xml

<dependency>
<groupId>org.apache.servicemix.bundles</groupId>
<artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
</dependency>

embedded these dependencies in depdencies or all pom.xml(according to your project structure)

<embedded>
<groupId>org.apache.servicemix.bundles</groupId>
<artifactId>org.apache.servicemix.bundles.kafka-clients</artifactId>
<target>/apps/<project-package>/application/install</target>
</embedded>