Batch Replication Functionality

[shivamra]

In my project I have to publish large amount of assets. So I enabled Batch Publishing option for my replication Agent so that it will create  batch of request instead of replicating assets individually. 

But the problem is when I replicate asset by calling replicator API from my servlet and provide it list of path. It is creating the batch request for each path I have passed to it Instead of clubbing all paths in one request as you can see it

I want to know whether it is expected behaviour or am I miss understood something . I have tried to pass 1100 paths to test this functionality and it is taking longer time to complete

[gkalyan]

@shivamra 

Unfortunately, Replicator API does not have direct control over batch replication settings. It treats each replication request as a separate entity, even when batch replication is enabled.

So, when you are using a servlet to programmatically send replication requests, the batch settings is taking each request a new batch request instead of packaging them as a single request.

Alternatively, there is older question community to handle high volume of activations. The second option suggested by @

markus_bulla_adobe in the below question might help you. 

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/aem-replication-agents-stu... 

[HrishikeshKa]

Hi @shivamra ,

The behavior you are experiencing might be due to the way the replication API processes the list of paths you provide. To better understand and potentially resolve this issue, let's go through a few key points and potential solutions.

Understanding Batch Replication

Batch replication is intended to group multiple replication requests into a single batch to optimize performance, especially when dealing with a large number of assets. This should theoretically reduce the number of individual requests and improve overall throughput.

Potential Issues and Considerations

1. API Call Implementation:

   • Ensure that when you call the replicator API, you are correctly passing the list of paths in a way that the API can interpret as a single batch request.
   • If each path is being processed as an individual request, it could be due to how the request is being constructed or how the API interprets it.

2. Replication Agent Configuration:

   • Verify the configuration settings of the replication agent to ensure it is set up to handle batch replication correctly. This includes batch size limits and other relevant parameters.

3. Code Implementation:

   • Review the servlet code that invokes the replication API to ensure it is correctly bundling the paths into a single batch request.

Steps to Diagnose and Fix the Issue

1. Review Replicator API Documentation:

   • Check the official documentation for the replicator API to ensure you are using it correctly for batch processing. Look for examples or specific parameters that might influence batch behavior.

2. Check Replication Agent Settings:

   • Ensure the replication agent's batch settings are correctly configured. There might be a setting that limits the batch size or alters how batches are created.

3. Example Code for Batch Replication:

   • 1. Logging and Debugging:

        • Add detailed logging to your servlet code to capture how the paths are being passed to the replicator API and how the replication agent processes them. This might help identify any discrepancies.

     2. Consult Support or Community:

        • If the issue persists, consider reaching out to the support team of the software you're using or participating in community forums. There might be known issues or additional configuration steps required.Here’s a generic example of how you might call the replication API to process a batch of paths:

     Batch replication should indeed club multiple paths into a single request, but if this is not happening, it might be due to misconfiguration or incorrect API usage. By reviewing the configuration, ensuring proper API usage, and debugging the code, you should be able to identify and resolve the issue. If all else fails, consulting official support or community forums can provide additional insights and solutions.

[shivamra]

Hi @HrishikeshKa Can you please provide any reference or article for programmatically send paths (using replicator API) for Batch replication. Since I have seen Adobe documentation related to it and it didnt provide much info on how to perform batch replication programmtically

[Jörg_Hoh · Accepted solution]

if your code is calling the replication API anway directly: There is replicator.replicate() method which takes an array of pathes to replicate. IIRC even in AEM 6.x this packaged these paths up in a single call (in AEM CS it definitely does that).