Error 500: javax.servlet.ServletException: For DiskStore: Replicated_UM_EMAIL_ALIAS_CACHE

[rajatkumar]

I'm running AEM Forms 6.5 JEE for WebSphere with the latest Service packs and hotfixes.
After running load/stress test on the server, I'm getting this error :

Due to which I'm unable to access my applications in AdminUI resulting my forms to not load.

Is this an expected behavior in heavy usage?
Can I delete the /tmp/adobews_* directory as a whole?  this is just 151mb?  What will be the impact of deleting the directory as a whole?

Note :- There is more than 5 GB of space left in the parent directory /app. still it is saying Critical disk usage.

Thank You in advance.

[Pulkit_Jain_]

@rajatkumar 

If you look closely, the msg is pointing to the caching directory i.e /tmp/adobews_*/caching/Data/Replicated** so this issue is specific to the GemFire DiskStore threshold.

As the error message says ,'Critical Disk Usage' the first step should be to check if the system has available space or not. If space is not available, delete some files and restart the server.

Previously, we have seen this issue raised by other customers, and the following changes were suggested by Pivotal to tackle this issue:

Add the following JVM arguments on the server:
-Dgemfire.statistic-sampling-enabled=true
-Dgemfire.statistic-archive-file=/app/gemfirestatslogs/mystatsFile.gfs
-Dgemfire.archive-file-size-limit=100
-Dgemfire.archive-disk-space-limit=1000

One of these parameters (-Dgemfire.statistic-archive-file) is for generating gemfire stats logs. Logs will be generated at the path specified, for reference I have used /app/gemfirestatslogs/mystatsFile.gfs, please make sure we have 'gemfirestatslogs' folder present under /app. You may change the path as per your convenience.

If the issue reappears even after adding these parameters, please raise a support ticket with the mystatsFile.gfs and the server logs so we can resolve this issue with Pivotal.

[rajatkumar]

@Pulkit_Jain_ 
Hi Pulkit, thanks for the prompt reply!
The whole issue started with one load/stress test in which observed that memory was not getting release in AEM resulting in heap/core dumps getting generated due to which disk space ran out.

So, followed this articled Data Store Garbage collector , ran the workflows and restarted the servers, which resolved the issue.
Although, if so is the case, will it always cause the same or we can setup something to clear out space in the background?

if the issue persists, will try adding these JVM parameters and see.

[Pulkit_Jain_ · Accepted solution]

@rajatkumar 

This case is very specific to implementation/load, and you may not get this issue that often. 

If the issue persists, add the JVM arguments and share the logs for further analysis. 

[Mayank_Gandhi]

@rajatkumar 

1. Browse to the AEM system console: http://host:port/lc/system/console
2. Locate the bundle

   org.apache.felix.webconsole.plugins.memoryusage

3. Stop the bundle
4. follow steps by pulkit