Additional publisher in architecture question

[P_V_Nair]

Can I get opinion on requests at the webserver level.?

If frequently we approach over 95 requests per second on the single webserver, do you think I can use this as a speaking point to the need to add the second server to provide ongoing resilience and handle increased volumes of traffic. 

Normal volumes are around 10 RPS.

Any suggestions would be appreciated on what scenario we go for an additional publish instance?

Our current architecture is one author, one publisher, one dispatcher with assets product. When we set the instance, it was around 1 million images in dam, but now it is around 4 million and looking for suggestions on how to claim for another publisher instance with the request per second stats we see in new relic now.

[Nitin_laad · Accepted solution]

Hi @P_V_Nair 

The high number of requests per second can cause performance issues, such as slow response time or even server downtime.

Adding a second server can help to distribute the load and ensure that the system can continue to handle the increased traffic. Additionally, having a second server as a failover option can also provide redundancy and ensure that the system remains available in case of any failures or issues with the primary server.

It's important to note that in order to support high traffic, the second server should be properly configured and optimized, and also it's important to consider other factors such as the content and data architecture, caching, and indexing.
It's recommended to consult with the performance engineer and system administrator to determine the best approach for your specific use case.

[P_V_Nair]

@Nitin_laad  Thank you for this detailed reply. But my main question was around 95 requests per second on one server. Is that something valid to point out to get the budget for a second server along with the context as a back up server?

[Shashi_Mulugu]

@P_V_Nair this is like " Not one Size Fit for all " type of question. It depends on various other factors such as memory, storage, cpu cores etc.. what you can do is, if you have similar setup in lower environments(ideally should like Stage/Uat/Perf), execute a perf run at >95 TPS and see the response times, server stats at each stage...Based on that it's a collective decision we need to make along with IT teams (assuming this is On-Prem).

Other recommendations:

Try adding a CDN on top of architectureture with sufficient Cache TTL so that AEM Dispatcher will not get somany hits.

Always best to have more than one publisher-Dispatcher combination as it helps during deployment strategy by making one set down, install code, test it, make them up, and proceed to set2 .. with this always one set is serving end user.

[Monendra_Singh]

Hi @P_V_Nair 

A single webserver can handle a certain amount of requests per second, and if your traffic is frequently approaching 95 requests per second on the single webserver, it suggests that the server is reaching its maximum capacity.

It's worth noting that the number of requests per second (RPS) that a web server can handle can vary depending on the resources available and the complexity of the requests. However, as a general rule, if you are frequently approaching 95 RPS on a single webserver, it would be beneficial to consider adding an additional web server to handle increased traffic.

An alternative way is to go for a CDN solution which would be less costly than maintaining and having another set of publisher instance. Having said that I can see you only have one publisher-disp setup in place so I would go for minimum 2 publisher Dispatcher setup given the volume of assets to be rendered.

Thanks

[BrijeshYadav]

Hi @P_V_Nair ,
With the limited information provided here its hard to provide anything concreate but lets talks about standard considerations.  Its not only the number of requests but also certain other factors also make significant performance difference. Like,

Does  aem page have other integrations to fetch the data to display ?
Does aem page is static or it more of dynamic in nature ?
How heavy is AEM page with dam assets ? Is it just 1 or 2 images on page or does it contains multiple high resolution images along with videos ?.
How long digital assets cached , are those changes very frequent or cached for longer time ?

I would also like to highlight here that with one Publisher and one Dispatcher its always a risk to handle fail over situation. So recheck your strategy here in case publisher or dispatcher fail over occur.

Regarding 4 million's images , Are all of these in use or just latest images used ? if this is the case then there should be a automate solution to remove old images periodically because maintaining such a huge amount of assets require certain expensive AEM internal operations which relates directly to AEM performance.