Heap dump analysis on aem server

Avatar

Avatar

apolu

Avatar

apolu

apolu

17-01-2020

Hi All,

 

Frequently we are observing one of the publisher is going down with java.lang.OutOfMemoryError: Java heap space error.

From the HeapDump analysis using Eclipse MAT tool, we got below Problem Suspects stack trace..

Problem Suspect 1

One instance of "org.apache.jackrabbit.oak.segment.CachingSegmentReader" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0" occupies 251,720,416 (11.62%) bytes. The memory is accumulated in one instance of "org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x7425fd5e0".

Keywords
org.apache.jackrabbit.oak.segment.CachingSegmentReader
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x7425fd5e0

Problem Suspect 2

One instance of "org.apache.jackrabbit.oak.cache.CacheLIRS" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610" occupies 220,491,096 (10.18%) bytes. The memory is accumulated in one instance of "org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610".

Keywords
org.apache.jackrabbit.oak.cache.CacheLIRS
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610

Details »

Problem Suspect 3

17,374 instances of "org.apache.jackrabbit.oak.segment.Segment", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0" occupy 231,538,184 (10.69%) bytes. These instances are referenced from one instance of "com.google.common.cache.LocalCache$Segment[]", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0"

Keywords
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0
com.google.common.cache.LocalCache$Segment[]
org.apache.jackrabbit.oak.segment.Segment
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0

 

Problem Suspect 4

15,729 instances of "org.apache.jackrabbit.oak.segment.Segment", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d86aba0" occupy 220,611,808 (10.18%) bytes. These instances are referenced from one instance of "com.google.common.cache.LocalCache$Segment[]", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0"

Keywords
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0
com.google.common.cache.LocalCache$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d86aba0
org.apache.jackrabbit.oak.segment.Segment

 

Can someone please go through above and provide me the suggestion what exactly caused from the above stack trace, i mean need detailed explanation to convey with dev team

 

Thanks,

ANR

View Entire Topic

Avatar

Avatar

Phoenics

Avatar

Phoenics

Phoenics

17-01-2020

Hello ANR,

 

The information you have shared below is insufficient. However based on  the information shared below i could suggest a few things,

 

1. As the issue you are facing is only on on instance & Not all . Hence it may be worth doing a revision cleanup  on this instance.

2. The Leak suspects don't generally reveal component / human readable issue information. A rather easier way of root cause analysis using the heapdumps is to  use the dominator tree which gives a much detailed level info on the component wise issue root cause.

 

Screenshot 2020-01-17 at 14.09.14.png