AEM6: DataStore growing so fast and DS Garbage Collection doesn't work (UnsupportedRepositoryOperationException) | Community
Skip to main content
O
oscar_maldonado
Level 2
October 16, 2015
Solved

AEM6: DataStore growing so fast and DS Garbage Collection doesn't work (UnsupportedRepositoryOperationException)

  • October 16, 2015
  • 3 replies
  • 1031 views

Hi!

I'm reposting my question in order to clarify our problem. 

We migrated to AEM6 from CQ5.6 one week ago. We followed the procedure detailed in https://docs.adobe.com/docs/en/aem/6-0/deploy/upgrade.html. To migrate the repository to OAK we used crx2oak-1.0.12.jar (indicated at that time). We set the datastore and the segmentstore (TarMK) according to the guide. After the first start up, we had this problem . So we installed SP2 directly. We installed also the hot fixes. Our tests were OK. Everything was perfect. 

However, one week later,  we noted that the publishers datastore folder had almost doubled its size. So we decided to run the datastore garbage collection and as soon as we invoke the task, an "UnsupportedRepositoryOperationException" is displayed (stack trace below). Giving that we have 2 publishers, we configured our load balancer to direct user traffic ONLY to one publisher. We found out that the publisher's datastore folder whose instance was removed from the load balancer stopped growing. 

We hope someone can give us some advice.

Thanks a lot.

javax.jcr.UnsupportedRepositoryOperationException

at com.day.crx.sling.server.impl.jmx.ManagedRepository.runDataStoreGarbageCollection(ManagedRepository.java:134) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75) at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at javax.management.StandardMBean.invoke(StandardMBean.java:405) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at com.adobe.granite.jmx.internal.JMXConsolePlugin.invoke(JMXConsolePlugin.java:176) at com.adobe.granite.jmx.internal.JMXConsolePlugin.doPost(JMXConsolePlugin.java:134) at javax.servlet.http.HttpServlet.service(HttpServlet.java:641) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.apache.felix.webconsole.internal.servlet.OsgiManager.service(OsgiManager.java:526) at org.apache.felix.webconsole.internal.servlet.OsgiManager.service(OsgiManager.java:450) at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:339) at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:300) at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:93) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:50) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.i18n.impl.I18NFilter.doFilter(I18NFilter.java:128) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.felix.http.sslfilter.internal.SslFilter.doFilter(SslFilter.java:55) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.felix.http.sslfilter.internal.SslFilter.doFilter(SslFilter.java:89) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at com.adobe.granite.license.impl.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:300) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.security.impl.ReferrerFilter.doFilter(ReferrerFilter.java:290) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.featureflags.impl.FeatureManager.doFilter(FeatureManager.java:115) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.engine.impl.log.RequestLoggerFilter.doFilter(RequestLoggerFilter.java:75) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:76) at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:49) at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745)
    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by oscar_maldonado

    Finally we solved our problem.

    1. We use datastore (external store) and the Linkchecker functionality was using it. So after changing its default configuration, our publishers stopped growing. See https://helpx.adobe.com/experience-manager/kb/AEM_6_TarMK_Online_Compaction.html  

    2. In order to reduce our datastore, we run "Offline Compaction via the Oak-run Tool" and finally datastore GC for aem6 (http://<host>:<port>/system/console/jmx/org.apache.jackrabbit.oak%3Aid%3D12%2Cname%3D%22repository+manager%22%2Ctype%3D%22RepositoryManagement%22)

    3 replies

    Sham_HC
    Sham_HC
    Level 10
    October 16, 2015

    By default aem6 system doesn't use a datastore, it stores everything in the tar files so you should not run datastore hc & also not worry about datastore gc.

    So instead, to reduce the size of your repository you can follow the steps under "Offline Compaction via the Oak-run Tool" on this page of the AEM docs:
    http://docs.adobe.com/docs/en/aem/6-0/deploy/upgrade/microkernels-in-aem-6-0.html
    Note that when you run offline compaction make sure to give it as large of a JVM heap size as possible. For example use the JVM parameter -Xmx8g

    O
    oscar_maldonadoAuthorAccepted solution
    Level 2
    October 16, 2015

    Finally we solved our problem.

    1. We use datastore (external store) and the Linkchecker functionality was using it. So after changing its default configuration, our publishers stopped growing. See https://helpx.adobe.com/experience-manager/kb/AEM_6_TarMK_Online_Compaction.html  

    2. In order to reduce our datastore, we run "Offline Compaction via the Oak-run Tool" and finally datastore GC for aem6 (http://<host>:<port>/system/console/jmx/org.apache.jackrabbit.oak%3Aid%3D12%2Cname%3D%22repository+manager%22%2Ctype%3D%22RepositoryManagement%22)

    O
    oscar_maldonadoAuthor
    Level 2
    October 16, 2015

    Thanks for your response!

    We tried the solution given but we couldn't solve the problem. Actually, we use datastore because it is suggested in the migration procedure.

    Thanks anyway.

    Terms & ConditionsAccessibility statement

    Loading footer...