Kafka Data won't show in AEP

[ChanuteJo]

Dear community,

I was tasked with connecting Streaming Event Data into AEP using Kafka. I was following "This Link" (https://experienceleague.adobe.com/en/docs/platform-learn/tutorial-one-adobe/activation/rtcdpb2c/rtc...) "Link 2" (https://experienceleague.adobe.com/en/docs/platform-learn/tutorial-one-adobe/activation/rtcdpb2c/rtc...) and "Link 3" (https://experienceleague.adobe.com/en/docs/platform-learn/tutorial-one-adobe/activation/rtcdpb2c/rtc...).

I was able to get to the very end of the exercise however, In Exercise 2.6.4 under "Produce an experience event" step, I ingest a test data as the exercise mentioned. However, upon checking, I was not able to see the test data I just uploaded into AEP. Upon further inspection, in Exercise 2.6.4 under "Create your Adobe Experience Platform Sink Connector using Postman" during the step where I had to use POSTMAN with the request of "GET Check Kafka Connect Connector Status". I was able to see that in the RESPONSE, "tasks" "state: FAILED" (The whole response is attached below" . I was wondering if this is the reason why I could not see the uploaded data into AEP?

On top of that, on the step "POST Create AEP Sink Connector", I will provide the REQUEST attached below

Regardless, please advice on what is the cause of this and how I could resolve the issue. Please do not hesitate to ask me for any further information and clarification.

Best regards,

Sching

[Parvesh_Parmar]

Hi @ChanuteJo , 

As I understand your flow is : 

[Your data source or application/ POST MAN] → [Kafka topic: aep] → [AEP Sink Connector] → [Adobe Experience Platform (AEP)]

Now based on the error log, the AEP Sink Connector task is failing, but that alone doesn't confirm whether the problem is before Kafka (data never reaches the topic) or after Kafka (data is invalid or causes failure in the connector).

Key Questions for Debugging:

1. Is the data actually reaching the Kafka topic?
   Use the following command to check:

   kafka-console-consumer --bootstrap-server localhost:9092 --topic aep --from-beginning
   

   If you see data, move to step 2. If not, the issue is with the producer or source connector that’s supposed to send data to Kafka.

2. If data is reaching Kafka, is it valid JSON?
   Your error message:

   com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 25))
   

   means the AEP Sink Connector tried to parse something not valid JSON — possibly binary data, malformed JSON, or control characters. This is what’s causing the task to fail.

Recommendation:

• First, verify if the data is in the aep Kafka topic.

• Then, check the format — make sure it’s clean JSON without control characters.

Hope this helps you isolate the issue more precisely!

Kr,

Parvesh

https://www.linkedin.com/in/parvesh-parmar/

[WarisaraYi]

Thanks again for the breakdown 

I’ve verified that data is reaching the Kafka topic (aep).
I also cleaned up the payload using jq -c . to ensure it's valid, single-line JSON, and I'm using schemas.enable=false.

Here’s the actual JSON I’m producing:

json

{"_wrkshp5snbxamer":{"accountId":"1234567890","amount":1000.5,"balanceAfterTransaction":2500.75,"channel":"mobile","locationIP":"192.168.1.1","locationOrIpAddress":"Bangkok","transactionAmount":1000.5,"transactionId":"txn-001","transactionStatus":"Success","transactionTimestamp":"2025-05-14T12:30:00Z"},"_id":"txn-001","eventMergedId":"merge-abc-001","eventType":"financialTransaction","identityMap":{"accountId":[{"id":"1234567890","primary":true,"authenticatedState":"authenticated"}]},"producedBy":"KrungsriCoreBank","timestamp":"2025-05-14T12:30:00Z"}

However, I’m still getting this error:

JsonEOFException: Unexpected end-of-input: expected close marker for Object

Could you advise what the next step should be from here?

[Supriya_Savre]

Hi @ChanuteJo,

Did the shared solution help you out? If yes, kindly consider marking the answer as ‘correct’.

If you’re still facing any challenges, please feel free to continue the conversation here. Or if you've discovered a solution yourself, we would appreciate it if you could share it with the community.

[kautuk_sahni]

@ChanuteJo Just checking in — were you able to resolve your issue?
We’d love to hear how things worked out. If the suggestions above helped, marking a response as correct can guide others with similar questions. And if you found another solution, feel free to share it — your insights could really benefit the community. Thanks again for being part of the conversation!