AEM author heap size error

Avatar

Avatar

narendiran100

Avatar

narendiran100

narendiran100

15-03-2021

We are using AEM 6.2 author instance and we ran into lot of performance issues lately. From the logs we found that the error is related to heap memory so we requested for a dump from server team so the we can analyse the memory leak issue

Hardware configurations:

 

Repository size: 250 GB

Free space : 400 GB

RAM: 64 GB

Heap was 12 GB and we decided to increase to 14GB as we assumed the out of memory is because of the heap size but still we faced slowness in AEM author and CPU load is increasing rapidly.

 

While we analyse the heap dump, I would like to know how to determine the size requirement of the RAM for the machine and Heap memory for AEM.

Also any suggestions/ links which we can follow to determine the root cause of slowness would be much appreciated. Please let me know if more details are required

Accepted Solutions (1)

Accepted Solutions (1)

Avatar

Avatar

KiranVedantam1992

Avatar

KiranVedantam1992

KiranVedantam1992

16-03-2021

Hi @narendiran100,

 

For hardware requirements please check this link

 

Regarding the system performance, please perform the maintenance activities listed here as per your requirement.

 

Also, regarding the performance, please check your code for the below issues.

  1. Reduce the session leaks
    1. You can achieve this by reducing the use of the JCR session
  2. Closing the ResourceResolver whenever it is opened
  3. Prefer the ResourceResolver over the JCR Session
  4. Do not share nodes, resources, and pages between threads and requests

Hope this helps.

 

Thanks,

Kiran Vedantam

Answers (2)

Answers (2)

Avatar

Avatar

Jörg_Hoh

Employee

Total Posts

3.1K

Likes

1.0K

Correct Reply

1.1K

Avatar

Jörg_Hoh

Employee

Total Posts

3.1K

Likes

1.0K

Correct Reply

1.1K
Jörg_Hoh
Employee

17-03-2021

Hi,

 

it's hard to give proper recommendations based on the given information. 12G of heap is plenty for most usecases, and not enough if you deal with very large assets. So increasing the heap size might or might not improve the situation, that totally depends on your application and your workload.

 

To understand the root cause for slowness, I would use the following approach:

* enable Garbage Collection logging on the JVM and monitor the GC behavior.

* quantify "slowness": Determine "slow" requests and make it actionable (what makes a slow request "slow"? Response times over 1 second?)

* check the periodic threaddump and search for patterns; correlate it with observed slowness (slow requests). It's definitely best if you could reproduce this in a lower environment and attach a profile to it to understand where the time is spend.

 

There are tons of ways, how and why requests get slow; a few of them:

* JCR queries, which are not served by an index (you will see warnings in the logs)

* overly complex logic with lots of unnecessary repository access

* requests, which trigger itself external requests to 3rd party systems.

 

HTH,

Jörg

 

Avatar

Avatar

narendiran100

Avatar

narendiran100

narendiran100

17-03-2021

Adding more information :

 

We are using below features in our instance

 

  • Lot of sling jobs running
  • Product data imported from xml and pages are created based on this. This happens during off business hours.
  • Asset ingestion is high both from content authors and through auto upload
  • Assets are auto published as we are autotagging them based on some business rules
  • As mentioned heap size  before heap size and direct memory allocated for AEM repository is  10 g earlier it was 8G -Xmx14g -Xms14g -XX:MaxDirectMemorySize=10g

I came across an article -https://dekesmith.info/index.php/2015/03/26/think-outside-the-heap-with-aem-6-and-jackrabbit-oak-tar...  which explains that segmenstore loads a part of it into direct memory so our segmenstore is 250 GB but having only 10G allocated in RAM is it fine ? Is there a thumb rule based on which we can balance this heap and directmemory for repository. In all articles I have checked I couldn't get more information on this.

 

Also we have total 64 GB of RAM and 8 CPU cores currently in the server and we have a spring application(for search)and solr also running in the same server. We understand if we change the heap then we may need to increase these parameters as well but I would like more inputs from experts here how to determine this.