Expand my Community achievements bar.

Get ready! An upgraded Experience League Community experience is coming in January.

How can we connect Kafka with adobe app builder

Avatar

Level 4

I have a requirement to integrate Kafka with the App Builder starter kit. Since Adobe Commerce event actions have a default execution time limit of 1 minute, I plan to use Kafka to store the event data temporarily. The idea is to publish the data to a specific Kafka topic and later consume it using a subscriber, as Kafka requires Docker to run locally.
Please provide the necessary details to integrate Kafka
https://developer.adobe.com/commerce/extensibility/events/tutorial/
Do we need an external service for Kafka to work with Adobe App Builder 

4 Replies

Avatar

Employee
Employee

Hi, I think you don't need to integrate with Kafka. You can use Adobe I/O events custom events to store the event data temporarily. Please look at this guide. https://developer.adobe.com/app-builder/docs/resources/event-driven/ 

Avatar

Level 4

Hi @tmj ,

Thanks for your response.

Yes, we do have alternative approaches available, such as using a custom event, triggering a webhook, or storing data in IO Files and processing it in smaller chunks. However, I mainly wanted to confirm whether Kafka can be integrated directly with Adobe App Builder. From what I understand, App Builder does not natively support running Kafka brokers or consumers within its runtime environment. If integration is possible, would it require an external service-such as a dedicated Kafka cluster or a middleware layer-to handle communication between Adobe App Builder and Kafka?

Avatar

Employee

Hi, yes, you can use a NodeJS library like KafkaJS in your code to connect to Kafka. 

Avatar

Level 3

@ShubhamAg2 

 

Must ensure the Kafka service is hosted on the following ports.
These are all the outbound connection ports that are allowed on the I/O Runtime.

21, 22, 25, 53, 80, 123, 143, [200-299], 389, 443, 445, 465, 587, 636, 1433, [2000-2999], 3000, 3306, 4242, 4317, 4343, 5400, 5432, 5671, 5672, 6061, 6062, 6379, 6380, 6651, 8000, 8020, 8080, 8085, 8088, 8089, 8300, 8500, 8600, 9090, 9092, 9093, 9094, [9096-9352], 9354, [10000-20000], 27016, 27017, 27018, 27019, 30303, 50010, and 60020.

 

To allow the additional ports, you can submit a ticket to the Support Team.