AEM takes significant high CPU and spikes when many requests are sent continously | Community
Skip to main content
E
extwebd96701854
Level 2
November 30, 2022
Solved

AEM takes significant high CPU and spikes when many requests are sent continously

  • November 30, 2022
  • 4 replies
  • 4507 views

Hi,

We've been experiencing performance issues on our AEM-backed websites for a while.

Recently, we've decided to investigate and found that, even on a fresh new vanilla AEM instance, the CPU usage still spikes when about 10-20 requests are sent continuously to any page on the sample content or the WKND project.

We've tried sending 20 requests per second for 10 seconds, which is equivalent to 200 users accessing the page in 10 seconds. This is not a high number. But the CPU usage still spikes to 100% and the server is inaccessible for more than 1 minute.

Is there any opinion to mitigate the performance issue?

The specs are as below:

  • Production-like Azure machine:
    • vCPUs 8
    • CPU Architecture x64
    • Memory 64 GiB
    • Hyper-V Generations V1, V2
    • Performance Score 151201
    • Processor AMD EPYC 7452 32-Core Processo
    • OS disk size 1023 GiB
    • Combined Write 128 MiB/Sec
    • Combined Read 128 MiB/Sec
    • OS Linux (RedHat 7.9)
  • Test machine:
    • 4 cores CPU Intel Core i7-8665U
    • Memory 32 GiB
    • OS Windows 11
  • Test machine:
    • 2 cores CPU
    • Memory 24 GiB
    • OS Linux (Fedora)

All of them experience the same CPU issue when running:

  • AEM service pack 6.5.14
  • With or without WKND 2.1.2
    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by joerghoh

    What is the result you would expect from this experiment? And do you think that this test is realistic for a production-like scenario?

     

    Any AEM deployment comes on the publish side with a dedicated caching layer (dispatcher) which should be used as much as possible. That means that rendering all requests directly on AEM should be avoided.

    4 replies

    TarunKumar
    Community Advisor
    TarunKumarCommunity Advisor
    Community Advisor
    November 30, 2022

    Hi @extwebd96701854 ,
    Can you please check if your caching mechanism is working fine in you case, this shouldn't be the case if cached files are rendered.

      Jagadeesh_Prakash
      Community Advisor
      Jagadeesh_PrakashCommunity Advisor
      Community Advisor
      November 30, 2022

      @extwebd96701854 as @tarunkumar said you should check if the content is getting served from cache or it might be hitting the publisher server every time

      Qucik check: how are you testing this? do you have the disaptcher on top of the publisher?

      And i am suprised why this is happening on vanilla instance. If your cache mechanisam is working fine then raise a adobe ticket and reachout to Adobe support team

        Avinash_Gupta_
        Community Advisor and Adobe Champion
        Avinash_Gupta_Community Advisor and Adobe Champion
        Community Advisor and Adobe Champion
        November 30, 2022

        Hi @extwebd96701854 

         

        High CPU utilization could be related to multiple issues. I would suggest you to analyze it from all the angles.

         

        • Check CPU utilization for any AEM process eating much memory. Collect thread dumps and analyze them.
        • Study the request.log file for detailed data.
        • Workflow checks, version checks, audit checks
        • Analyze caching strategies implemented at the AEM dispatcher
        • Check maintenance related to Oak repositories
        • Check indexing and review the jcr queries

         

        Hope this helps.

          joerghoh
          Adobe Employee
          joerghohAdobe EmployeeAccepted solution
          Adobe Employee
          December 3, 2022

          What is the result you would expect from this experiment? And do you think that this test is realistic for a production-like scenario?

           

          Any AEM deployment comes on the publish side with a dedicated caching layer (dispatcher) which should be used as much as possible. That means that rendering all requests directly on AEM should be avoided.

            E
            extwebd96701854Author
            Level 2
            January 20, 2023

            Hey @joerghoh 

            What is the recommendation in the case where we do not have static content that can be cached, such as resources are rendered based on permissions of users accessing the solution. We do cache whatever resources is possible on both dispathcer and CDN (in our Production setup). But still experience this challenges during peak traffic

             

             

            joerghoh
            Adobe Employee
            joerghohAdobe Employee
            Adobe Employee
            January 27, 2023

            In this case this is kind of expected; on the other hand side even in this scenario the rendering is not completely personalized; typically large parts of the results are identical (for example page navigation or other aspects).
            You can make use of this and split the pages into static and dynamic parts, and just render the dynamics always on the publish; Sling Dynamics Include is often used for this.

            Terms & ConditionsAccessibility statement

            Loading footer...