Publish Approach

[arvind]

Hi Team,

We have one Cron job scheduler which creates around 100k pages in Production (weekly twice) and then we need to publish all pages.

Would like to know best approach to handle this situations so that It does not impact Prod AEM instances' performance.

1) Page by page publish OR

2) Create Package through APIs and then replicate. OR

3) OR any best approach 

Regards,

AP

[Anish-Sinha]

Hi @arvind ,
As you mentioned, for 100k pages, page by page publish is difficult. Also creating the package and then replicate would be an additional step. The best here you could do is bulk replication for batches. For example, since you have the list of all the pages, you can pick 500/1000 pages per batch based on the performance you see on server and do bulk replicate, once the task is finished, take the next batch. This would not impact your performance and it will not stuck in the queue. 

[arvind]

Thank you Anish.

Do we have any Bulk replicate API in place to use at code level?

Regards,

Arvind

[Anish-Sinha · Accepted solution]

As mentioned by Shubham, you can use :

replicate(Session session, ReplicationActionType type, java.lang.String[] paths, ReplicationOptions options)

Sample class here: https://github.com/Adobe-Consulting-Services/acs-aem-commons/blob/master/bundle/src/main/java/com/ad...

get API details from here. 

[arvind]

Thank you.

[Shubham_borole]

@arvind 

100k pages is a large number for a single run. Is there a specific structure to this? So that it can be divided based on parents? For better performance it is suggested in AEM node repository, the child nodes should not exceed more than 1000 See this & this .1000 is not a hard number as such. If you come up with such structure or set up to create pages in such manner, when you create the pages you can create say 1000 pages, save, and then replicate using https://www.adobe.io/experience-manager/reference-materials/6-5/javadoc/com/day/cq/replication/Repli... 

replicate(Session session, ReplicationActionType type, java.lang.String[] paths, ReplicationOptions options)

Then do some validation and proceed.
Please check for any existing preprocessor logic in your code and replication based event listeners. If not applicable the new pages section can be excluded to make sure there are lesser unwanted performance impacts.

Is there any clean up process in plans? In long term there will be lot of pages in your publish instances  

[arvind]

Thank you for your help.