Sending Data to Kafka with Prime License using AJO

[ChanuteJo]

Dear Community,

I would like to ask for the best recommendations using AJO for my client requirement. So in AEP, I created two schema, Individual Profile Class, and ExperienceEvent Class. What I want to achieve is creating an Audience in AEP, then using AJO to send ExperienceEvents field to Kafka. The limitation here is that the client only has Prime License. 

I understand that to send data to Kafka from AJO is through using Custom Action with the appropriate Payload. Currently, I have created and Audience, which is then ready to be used, then creating a Unitary Event. I create a Unitary Event because this allows me to select ExperienceEvent fields to map with the Custom Action. In test mode (Only has Unitary Event and Custom Action in the Canvas) I was able to get the notification that the test data is successfully sent and was able to see the data in the notification (it was sent to my phone). However, after publishing it, I could not get the Journey to work, sometimes the Journey just stopped before reaching custom actions, sometimes it went to the END stage but I did not receive any notification.

Could any suggest where I went wrong and how I could come up with a solution?

Please do not hesitate to ask for additional information

Best regards,
Sching

[SatheeskannaK]

@ChanuteJo It sounds like you're encountering an issue where your journey, configured with a unitary event and a custom action, isn't successfully sending event data payloads to your Kafka endpoint. Please validate the following to ensure everything is working as expected.

1. Event Schema and Profiles:

         Ensure your event schema and identity resolving properly against the Profile Store. 

• Schema Validation: Double-check that the schema of the unitary event you're sending exactly matches the expected schema in Journey Optimizer. Any discrepancies in field names, data types, or required fields can prevent proper resolution.
• Identity Resolution: Verify that the profile identifier used in your event configuration is correctly mapping to an existing identity in the Profile Store. This typically involves ensuring the _id field (or your designated primary identifier) in your event payload aligns with how profiles are identified in Adobe Experience Platform (AEP).
• Namespace Configuration: Confirm that the identity namespace used for your profile identifier is correctly configured and associated with your dataset in AEP. 

2. Triggering the Unitary Event via Postman: Utilize a tool like Postman to trigger the unitary event.
   • HTTP API Endpoint: Ensure you are using the correct HTTP API endpoint provided by Adobe Experience Platform to stream your event data. This endpoint is distinct from your Kafka endpoint and acts as the initial ingestion point.

• Payload Structure: Construct your event payload in Postman, making sure it adheres to the expected JSON structure and includes all necessary fields, especially your profile identifier.
3. Querying Journey Step Events for Errors:

Refer to the Adobe Experience League documentation for query examples related to journey step events (refer to this - How many errors occurred on each node of a specific journey for a certain amount of time query sample):

https://experienceleague.adobe.com/en/docs/journey-optimizer/using/reporting/reports/query-examples

• Key Error Fields to Investigate: When querying the journey step events, pay close attention to the following fields for failures:
• actionExecutionError: This field will provide a general description of any error encountered during the execution of your custom action.
• actionExecutionErrorCode: A specific numerical code associated with the error, which can often be cross-referenced with documentation for more detailed explanations.
• actionExecutionOriginError: Describes errors originating from the destination system (your Kafka endpoint in this case). This is particularly useful for identifying issues with your Kafka connection or data ingestion.
• actionExecutionOriginCode: A specific code from the destination system, offering further detail on the nature of the error from Kafka's perspective.
• Understanding Execution Fields: For a deeper dive into the available execution fields and their meanings, consult this page:
• https://experienceleague.adobe.com/en/docs/journey-optimizer/using/reporting/reports/legacy-step-eve...

[ChanuteJo]

1. The data is correctly Mapped to the Profile. The namespace are correctly configured, as well as in Custom Action Payload and mapping. Journey are also configured as according to AEP.

2. I installed Kafka on my local machine to ingest ExperienceEvent Data. Once ingested, the Event Data is correctly mapped to the associated profile.

3. I cannot use queries in the sandbox as the client only has PRIME License.

Please let me know how I could troubleshoot further or where I went wrong. If you need any further information, feel free to let me now.

Best regards,
Sching

[SatheeskannaK]

@ChanuteJo Understood. Let's try a specific approach to test this. Try to simulate an event trigger originating from a local Apache Kafka instance. This event, once triggered, should ultimately send a payload to a public endpoint via a custom action via journey. For the public endpoint, use anything like pipedream, or pokapi.co, or pipedream.

[Mayank_Gandhi]

@ChanuteJo the test mode in AJO uses simulated data – actual journeys rely on live ExperienceEvent ingestion and qualification. To begin with check the logs and see if the journey:

• Stalls at the event node (due to missing/incorrect event payload)
• Qualifies the audience, but stalls after Custom Action (could be HTTP/connectivity error)
• Check for "Event not received" or "Profile not qualified" errors.
• Try with a temporary public HTTP endpoint (like webhook.site) to validate connectivity.
  
  
  

[ChanuteJo · Accepted solution]

So the Solution to achieving this is as the following logic

Unitary events -> Wait (2 Minutes) -> Conditions (inAudience = True) -> Custom Action.

The reason I could not receive notification is due to the fact that the URL Endpoint I provided has some internal error in sending Notification to my phone.