Level 3

Solved

Impact on Dispatcher cache when 10k pages are deleted

Forum|Forum|4 months ago
September 17, 2025
5 replies
1086 views

Hi,
As part of optimizing authoring activity , business is looking to deactivate and delete 10k pages on prod author using groovy script.
We think this might impact cache flush on dispatcher.
As 10k cache flush request will be created.
Is there any suggestion to better handle this part ?
Thanks,

Best answer by giuseppebaglio

You could split the Operation into Batches: Instead of processing all 10,000 pages at once, break the operation into smaller batches (e.g., 100 or 500 pages per batch): it reduces the number of simultaneous cache flush requests and minimizes the risk of overwhelming the dispatcher or author instance. Besides you can add delays, ie introducing a small delay between batches to allow the system to process cache flushes and other background tasks.

Points of attention:

It is crucial to run the script during off-peak hours to minimise the impact on end-users and reduce the load on the dispatcher.
Keep an eye on system performance (CPU, memory, response times) during the script execution. If the system shows signs of stress, pause the script and resume later.
Before running the script in production, test it in a staging environment with a similar number of pages. This helps you gauge the impact and fine-tune batch sizes and delays.

TarunKumar

Community Advisor

Hi @ashish_mishra1 ,

You can try to implement a Custom Flushing Strategy. For a large volume of deactivations, avoid a full cache flush.
Instead, use the CQ:Path header in the replication agent's configuration to specify the exact paths that need to be invalidated.
This targets only the affected content and minimizes impact on the rest of the cache.

Adjust statfileslevel:Configure the /statfileslevel property in the dispatcher.any file.
This setting determines the number of parent nodes for which .stat files are checked.
A lower level reduces the impact of flush requests on the dispatcher by ensuring that only necessary files are invalidated.

Hope it helps!

-Tarun

giuseppebaglioAccepted solution

Level 10

You could split the Operation into Batches: Instead of processing all 10,000 pages at once, break the operation into smaller batches (e.g., 100 or 500 pages per batch): it reduces the number of simultaneous cache flush requests and minimizes the risk of overwhelming the dispatcher or author instance. Besides you can add delays, ie introducing a small delay between batches to allow the system to process cache flushes and other background tasks.

Points of attention:

It is crucial to run the script during off-peak hours to minimise the impact on end-users and reduce the load on the dispatcher.
Keep an eye on system performance (CPU, memory, response times) during the script execution. If the system shows signs of stress, pause the script and resume later.
Before running the script in production, test it in a staging environment with a similar number of pages. This helps you gauge the impact and fine-tune batch sizes and delays.

arunpatidar

Community Advisor

I agree with @giuseppebaglio
I would block the end user access to those pages first from CDN and then delete the pages in batches after business hours.

Arun Patidar

A

ashish_mishra1Author

Level 3

Thanks, @shashi_mulugu @arunpatidar @rohan_garg @tarunkumar @giuseppebaglio @joerghoh

Thanks for your replies, this monday we did tried to delete in 1k batches, but we could see the AEM Author CPU usage was quite high > 60% , and it took us 1 hour to delete 1k pages, which is quite slow. in comparison to stage where we tested 2k page deletion in 10 minutes.

One of the cause we think is the number of invalidation request going to publishers 1 and 2 and it slowing down the AEM author processing. This we performed in off business hours with no business activity.

Code used -

                        def deactivated = false
                        try {
                            // First Deactivate the page
                            replicator.replicate(session, ReplicationActionType.DEACTIVATE, page.path)
                            deactivated = true
                            println("Page Deactivated :: " + page.path)

                        } catch(Exception e) {
                            println("Failed to deactivate: " + page.path + ", error: " + e.message)
                        }
                        if (deactivated) {
                            // Then Delete the page if deactivation was successful
                            pageManager.delete(page, false)
                            deletedPages++
                            println("Page Deleted :: " + page.path)

                            if(deletedPages % 1024 == 0) {
                                println("Committing the changes at deleted count :: "+ deletedPages)
                                session.save()
                            }
                        }else{
                            println("Skipping deletion as deactivation failed for :: " + page.path)
                        }

Now, here we were using the PageManager API and then deleting it.

Any idea what going wrong?

Other option we could think of using JCR API to delete the page as Node object. This will not trigger the replication action, but we have to run it 1 author, 2 publishers separately and then flush cache.

we were wondering will there be any adverse impact of this on the AEM author internal references or indexes ?

I have raised this with Adobe support too, and waiting for their reply

Rohan_Garg

Community Advisor

Hi @ashish_mishra1,

Divide and conquer - Break the pages into batches (say 500) and schedule them during the off hours.
This should help in spreading out the flush events at Dispatcher.

Besides the stat file strategy mentioned by @tarunkumar, you can also use CDN purge if you have a CDN (Akamai, CloudFront etc.) you can flush cache at folder level. Targeting high level purges which would be handful would be better than having 10k small invalidations.
Are you having OOTB Fastly in AEM Cloud or some other CDN in your setup?

Hope this helps!

Best Regards,

Rohan Garg

Shashi_Mulugu

Community Advisor

@ashish_mishra1 it all depends on your current statfilelevel settings and at what levels your 10k pages are.

Please take others suggestions into considerations like deactivating during off business hours, stop live traffic to those pages first, deactivating in batches etc

My two cents:

If the impact of flushing is huge, we can do two things

1. Configure a agent user ID in publisher dispatcher agent, and add a deny read permissions for those 10k pages or if they are under some high level roots at those root levels, so that when you activate/deactivate those pages dispatcher agent wont pickup those events.

2. You can do similar in your dispatcher configuration, publish farm cache in validation rules on which paths or extensions to be allowed to get flushed. This is one step delay stop than #1 where 10k requests comes to dispatcher but dispatcher wont act as per rules.

I would go with #1.

Syed_Shaik

Level 3

Hi @ashish_mishra1 ,

To handle this more efficiently, here are some best practices and recommendations:

Batch the Deactivation and Deletion:
- Instead of running the Groovy script for all 10k pages at once, process them in small batches (e.g., 200–500 pages per batch).
- Add a delay between batches to reduce load on the Publish and Dispatcher tiers.
Temporarily Disable Dispatcher Flush Replication Agent (Optional):
- You can temporarily disable the flush agent while deactivating/deleting pages in bulk.
- After the operation, perform a manual cache invalidation or use a path-based flush (e.g., /content/site/en/section) to clear affected areas at once instead of 10k individual requests.
Leverage Content Path-Based Invalidation:
- If the pages belong to a specific subtree (e.g., /content/site/en/news/), consider using dispatcher invalidation rules or cache root deletion for that subtree, which is much faster than individual flushes.
Schedule During Off-Peak Hours:
- Run the bulk operation during low traffic windows to minimize impact on both authors and end-users.
Validate in Lower Environments First:
- Always test the performance and cache impact in Stage/UAT before executing in Production.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded