Expand my Community achievements bar.

Don’t miss the AEM Skill Exchange in SF on Nov 14—hear from industry leaders, learn best practices, and enhance your AEM strategy with practical tips.
SOLVED

Purge Versions tool timeout and OutOfMemory

Avatar

Level 4

Hi,

I am trying to purge versions on my AEM6.0+SP3 instance. When I run the tool even an a very small portion of the content tree I get this error from Web Logic. Somehow the process times out or fails and I am not able to purge versions. Second time I tried to run the version purge I have an OutOfMemory.

Do you know how I can troubleshoot this problem?

####<Nov 23, 2015 11:59:02 AM CET> <Info> <Health> <aem_ms1> <weblogic.GCMonitor> <<anonymous>> <> <> <1448276342926> <BEA-310002> <14% of the total memory in the server is free.>
####<Nov 23, 2015 12:13:42 PM CET> <Error> <WebLogicServer> <aem_ms1> <[STANDBY] ExecuteThread: '10' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1448277222014> <BEA-000337> <[STUCK] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "609" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 609916 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace:
        org.apache.jackrabbit.oak.plugins.segment.Segment.internalReadRecordId(Segment.java:326)
        org.apache.jackrabbit.oak.plugins.segment.Segment.readRecordId(Segment.java:322)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplateId(SegmentNodeState.java:70)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:79)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getProperty(SegmentNodeState.java:123)
        org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.getProperty(MemoryNodeBuilder.java:464)
        org.apache.jackrabbit.oak.core.SecureNodeBuilder.getProperty(SecureNodeBuilder.java:184)
        org.apache.jackrabbit.oak.plugins.tree.AbstractTree.getProperty(AbstractTree.java:226)
        org.apache.jackrabbit.oak.core.MutableTree.getProperty(MutableTree.java:116)
        org.apache.jackrabbit.oak.query.ast.SelectorImpl.currentOakProperty(SelectorImpl.java:633)
        org.apache.jackrabbit.oak.query.ast.SelectorImpl.currentOakProperty(SelectorImpl.java:604)
        org.apache.jackrabbit.oak.query.ast.SelectorImpl.currentOakProperty(SelectorImpl.java:537)
        org.apache.jackrabbit.oak.query.ast.SelectorImpl.currentProperty(SelectorImpl.java:513)
        org.apache.jackrabbit.oak.query.ast.PropertyValueImpl.currentProperty(PropertyValueImpl.java:106)
        org.apache.jackrabbit.oak.query.ast.ComparisonImpl.evaluate(ComparisonImpl.java:88)
        org.apache.jackrabbit.oak.query.ast.AndImpl.evaluate(AndImpl.java:123)
        org.apache.jackrabbit.oak.query.ast.SelectorImpl.next(SelectorImpl.java:431)
        org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.fetchNext(QueryImpl.java:659)
        org.apache.jackrabbit.oak.query.QueryImpl$RowIterator.hasNext(QueryImpl.java:679)
        com.google.common.collect.Iterators$5.hasNext(Iterators.java:542)
        org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.fetchNext(FilterIterators.java:137)
        org.apache.jackrabbit.oak.query.FilterIterators$DistinctIterator.hasNext(FilterIterators.java:151)
        org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$3.fetch(QueryResultImpl.java:181)
        org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$3.next(QueryResultImpl.java:207)
        org.apache.jackrabbit.oak.jcr.query.QueryResultImpl$3.next(QueryResultImpl.java:170)
        org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate$SynchronizedIterator.next(SessionDelegate.java:697)
        org.apache.jackrabbit.oak.jcr.query.PrefetchIterator.next(PrefetchIterator.java:97)
        org.apache.jackrabbit.commons.iterator.RangeIteratorAdapter.next(RangeIteratorAdapter.java:152)
        org.apache.jackrabbit.commons.iterator.RangeIteratorDecorator.next(RangeIteratorDecorator.java:92)
        org.apache.jackrabbit.commons.iterator.NodeIteratorAdapter.nextNode(NodeIteratorAdapter.java:80)
        com.day.cq.wcm.core.impl.VersionManagerImpl.getTreePurgeList(VersionManagerImpl.java:549)
        com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:442)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp$Processor.internalProcess(POST_jsp.java:154)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp$Processor.process(POST_jsp.java:130)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp._jspService(POST_jsp.java:409)

GC overhead limit exceeded

Cannot serve request to /etc/versioning/purge.html in /libs/wcm/core/components/versionpurge/POST.jsp

Exception:

java.lang.OutOfMemoryError: GC overhead limit exceeded
    at sun.nio.cs.UTF_8.newDecoder(UTF_8.java:68)
    at java.lang.StringCoding.decode(StringCoding.java:213)
    at java.lang.String.<init>(String.java:451)
    at java.lang.String.<init>(String.java:503)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.loadString(Segment.java:352)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:338)
    at org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:332)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:167)
    at org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:137)
    at org.apache.jackrabbit.oak.spi.query.PropertyStateValue.getValue(PropertyStateValue.java:61)

1 Accepted Solution

Avatar

Correct answer by
Employee

How much memory have you allocated?[0]

Regards,

Opkar

[0]https://docs.adobe.com/docs/en/aem/6-0/deploy/application-server-install.html#Oracle WebLogic 10.3.5 / 10.3.6 / 12.1

View solution in original post

3 Replies

Avatar

Correct answer by
Employee

How much memory have you allocated?[0]

Regards,

Opkar

[0]https://docs.adobe.com/docs/en/aem/6-0/deploy/application-server-install.html#Oracle WebLogic 10.3.5 / 10.3.6 / 12.1

Avatar

Level 4

-Xms5120m

-Xmx5120m

-XX:MaxDirectMemorySize=5120m

-XX:PermSize=512m

-XX:MaxPermSize=512m 

Total memory is 12G.

After re-executing the version purge I have no OutOfMemory anymore but I have an infinite loop when purging home page versions...It just cycle on the same N versions, so WebLogic reports that thread as STUCK.

Avatar

Level 4

This is the STUCK thread:

####<Nov 23, 2015 2:51:57 PM CET> <Error> <WebLogicServer> <aem_ms1> <[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1448286717050> <BEA-000337> <[STUCK] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "685" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 685521 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace:
        org.apache.jackrabbit.oak.plugins.segment.file.TarReader.findEntry(TarReader.java:585)
        org.apache.jackrabbit.oak.plugins.segment.file.TarReader.readEntry(TarReader.java:537)
        org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:822)
        org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:134)
        org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:101)
        org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
        org.apache.jackrabbit.oak.plugins.segment.MapRecord.size(MapRecord.java:128)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:461)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:459)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:459)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:711)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1089)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1091)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1072)
        org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1063)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1072)
        org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
        org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1063)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:91)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:26)
        org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:57)
        org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
        org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(CompositeHook.java:60)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:352)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:383)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:439)
        org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:151)
        org.apache.jackrabbit.oak.spi.state.ProxyNodeStore.merge(ProxyNodeStore.java:42)
        org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:247)
        org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:341)
        org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:366)
        org.apache.jackrabbit.oak.jcr.version.ReadWriteVersionManager.removeVersion(ReadWriteVersionManager.java:253)
        org.apache.jackrabbit.oak.jcr.delegate.VersionManagerDelegate.removeVersion(VersionManagerDelegate.java:225)
        org.apache.jackrabbit.oak.jcr.delegate.VersionHistoryDelegate.removeVersion(VersionHistoryDelegate.java:211)
        org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl$11.performVoid(VersionHistoryImpl.java:240)
        org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:268)
        org.apache.jackrabbit.oak.jcr.version.VersionHistoryImpl.removeVersion(VersionHistoryImpl.java:236)
        com.day.cq.wcm.core.impl.VersionManagerImpl.purgeVersions(VersionManagerImpl.java:452)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp$Processor.internalProcess(POST_jsp.java:154)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp$Processor.process(POST_jsp.java:130)
        org.apache.jsp.libs.wcm.core.components.versionpurge.POST_jsp._jspService(POST_jsp.java:409)

 

Thank you!