Hi All,
Frequently we are observing one of the publisher is going down with java.lang.OutOfMemoryError: Java heap space error.
From the HeapDump analysis using Eclipse MAT tool, we got below Problem Suspects stack trace..
Problem Suspect 1
One instance of "org.apache.jackrabbit.oak.segment.CachingSegmentReader" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0" occupies 251,720,416 (11.62%) bytes. The memory is accumulated in one instance of "org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x7425fd5e0".
Keywords
org.apache.jackrabbit.oak.segment.CachingSegmentReader
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x7425fd5e0
Problem Suspect 2
One instance of "org.apache.jackrabbit.oak.cache.CacheLIRS" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610" occupies 220,491,096 (10.18%) bytes. The memory is accumulated in one instance of "org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]" loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610".
Keywords
org.apache.jackrabbit.oak.cache.CacheLIRS
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d76f610
Details »
Problem Suspect 3
17,374 instances of "org.apache.jackrabbit.oak.segment.Segment", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0" occupy 231,538,184 (10.69%) bytes. These instances are referenced from one instance of "com.google.common.cache.LocalCache$Segment[]", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0"
Keywords
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0
com.google.common.cache.LocalCache$Segment[]
org.apache.jackrabbit.oak.segment.Segment
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x743115ad0
Problem Suspect 4
15,729 instances of "org.apache.jackrabbit.oak.segment.Segment", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d86aba0" occupy 220,611,808 (10.18%) bytes. These instances are referenced from one instance of "com.google.common.cache.LocalCache$Segment[]", loaded by "org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0"
Keywords
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x74301f6f0
com.google.common.cache.LocalCache$Segment[]
org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @ 0x76d86aba0
org.apache.jackrabbit.oak.segment.Segment
Can someone please go through above and provide me the suggestion what exactly caused from the above stack trace, i mean need detailed explanation to convey with dev team
Thanks,
ANR
Solved! Go to Solution.
Views
Replies
Total Likes
MAT did not point to the right problems. org.apache.jackrabbit.oak.segment.CachingSegmentReader is a standard AEM activity. It constantly accesses data*.tar files, hence MAT pointed to those the most common operations.
Your heap dump needs a deeper analysis. You need to open a support case.
Views
Replies
Total Likes
Hello ANR,
The information you have shared below is insufficient. However based on the information shared below i could suggest a few things,
1. As the issue you are facing is only on on instance & Not all . Hence it may be worth doing a revision cleanup on this instance.
2. The Leak suspects don't generally reveal component / human readable issue information. A rather easier way of root cause analysis using the heapdumps is to use the dominator tree which gives a much detailed level info on the component wise issue root cause.
Views
Replies
Total Likes
Hello @Phoenics,
You told that dominator tree can say much more than these errors:
We've checked the dominator tree:
Could you please suggest or tell if there something understandable what area should be fixed?
Thanks in advance.
Views
Replies
Total Likes
MAT did not point to the right problems. org.apache.jackrabbit.oak.segment.CachingSegmentReader is a standard AEM activity. It constantly accesses data*.tar files, hence MAT pointed to those the most common operations.
Your heap dump needs a deeper analysis. You need to open a support case.
Views
Replies
Total Likes
Hi @berliant, thanks for the update, we found memory consumption exhausted due to large size of images uploaded into system and the details we got from error.log and as well some clue from dominator tree view shows the largest objects in the heap dump.
Thanks,
ANR
Views
Replies
Total Likes
Views
Likes
Replies
Views
Likes
Replies