We have an AEM server that stopped responding to requests. It also stopped logging anything--the last log in crx-quickstart/logs was touched 8 hours ago (audit.log).
The last log in error.log is as follows:
10.08.2017 03:19:20.742 *INFO* [jackrabbit-pool-1] org.apache.jackrabbit.core.query.lucene.IndexMerger merged 22 documents in 10 ms into _w6f1.
10.08.2017 03:20:02.330 *INFO* [pool-6-thread-5] org.apache.jackrabbit.core.persistence.bundle.AbstractBundlePersistenceManager cachename=crx.defaultBundleCache[ConcurrentCache@6533291], elements=3590, usedmemorykb=8191, maxmemorykb=8192, access=511048, miss=87021
The last log in gc.log is as follows:
Heap after GC invocations=692 (full 1):
PSYoungGen total 1020864K, used 21552K [0x00000007bdeb0000, 0x0000000800000000, 0x0000000800000000)
eden space 959040K, 0% used [0x00000007bdeb0000,0x00000007bdeb0000,0x00000007f8740000)
from space 61824K, 34% used [0x00000007f8740000,0x00000007f9c4c020,0x00000007fc3a0000)
to space 59264K, 0% used [0x00000007fc620000,0x00000007fc620000,0x0000000800000000)
ParOldGen total 2165440K, used 1812875K [0x0000000739c00000, 0x00000007bdeb0000, 0x00000007bdeb0000)
object space 2165440K, 83% used [0x0000000739c00000,0x00000007a8662f00,0x00000007bdeb0000)
PSPermGen total 786432K, used 144799K [0x0000000709c00000, 0x0000000739c00000, 0x0000000739c00000)
object space 786432K, 18% used [0x0000000709c00000,0x0000000712967f78,0x0000000739c00000)
}
{Heap before GC invocations=693 (full 1):
PSYoungGen total 1020864K, used 980592K [0x00000007bdeb0000, 0x0000000800000000, 0x0000000800000000)
eden space 959040K, 100% used [0x00000007bdeb0000,0x00000007f8740000,0x00000007f8740000)
from space 61824K, 34% used [0x00000007f8740000,0x00000007f9c4c020,0x00000007fc3a0000)
to space 59264K, 0% used [0x00000007fc620000,0x00000007fc620000,0x0000000800000000)
ParOldGen total 2165440K, used 1812875K [0x0000000739c00000, 0x00000007bdeb0000, 0x00000007bdeb0000)
object space 2165440K, 83% used [0x0000000739c00000,0x00000007a8662f00,0x00000007bdeb0000)
PSPermGen total 786432K, used 144799K [0x0000000709c00000, 0x0000000739c00000, 0x0000000739c00000)
object space 786432K, 18% used [0x0000000709c00000,0x0000000712967f78,0x0000000739c00000)
1097170.030: [GC
Desired survivor size 57868288 bytes, new threshold 1 (max 15)
[PSYoungGen: 980592K->23456K(1026176K)] 2793467K->1836915K(3191616K), 0.0926920 secs]
Disk usage:
df -k .
Filesystem 1K-blocks Used Available Use% Mounted on
30800828 20815900 8418196 72%
The "status" script hangs:
./status
10.08.2017 15:52:14.147 *INFO * [main] Setting sling.home=. (command line)
I believe restarting the AEM server will fix the issue, but it would be good to know what is happening and how we can prevent it from happening in the future. Any help would be appreciated.
