Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
Bedrock Mission!

Learn more

View all

Sign in to view all badges

SOLVED

Thread Dump and Heap Dump issues

GleniB
Level 1
Level 1

Good Morning,

 

Our publisher servers seem to be having a very hard time with the load recently. We were analyzing the thread dumps to see if we could find anything unusual and we saw that we had A LOT of waiting and blocked threads. All the waiting threads give this message:
- waiting to lock <0x3b9b994f> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) owned by "null" tid=0x-1

After some research we found that this could be caused by garbage collection taking up resources as it is trying to clear up space.
We tried analyzing the heap dumps, but full honesty we did not know exactly what to look for in them.

Has anyone seen this before or have any advice on how we can proceed to troubleshoot this issue?
Thank you!

1 Accepted Solution
PulkitJain
Correct answer by
Employee
Employee

@GleniB 

I think the first step would be to review the thread dump thoroughly for any stuck/blocked threads on any particular process/activity, since you've mentioned "having a very hard time with the load recently".

In case of OutOfMemory situations OR frequent garbage collection activities (drawing conclusion from thread dump analysis), you can review the Heap dumps to check for any memory leakage, objects with high memory usage or object graph to identify objects of not releasing memory.

View solution in original post

3 Replies
PulkitJain
Employee
Employee

@GleniB 

Garbage collection happens every now and then on the server but if the blocking thread belongs to a doamon thread e.g. garbage collection thread then it could be an issue. Its worth reviewing the heap dumps[0] as well to check why is the garbage collection happening so frequently. 

 

For a quicker turnaround, please raise a support ticket along with the required details/dumps.

 

[0] - https://heaphero.io/ 

GleniB
Level 1
Level 1

Thank you, 
Is there anything in specific we should be looking for in the heap dump?

PulkitJain
Correct answer by
Employee
Employee

@GleniB 

I think the first step would be to review the thread dump thoroughly for any stuck/blocked threads on any particular process/activity, since you've mentioned "having a very hard time with the load recently".

In case of OutOfMemory situations OR frequent garbage collection activities (drawing conclusion from thread dump analysis), you can review the Heap dumps to check for any memory leakage, objects with high memory usage or object graph to identify objects of not releasing memory.

View solution in original post