Expand my Community achievements bar.

SOLVED

TarMK Flush Thread

Avatar

Level 8

I've noticed when replicating large amounts of data, or installing large packages, we see the following in our error logs:

10.07.2014 08:18:11.045 *INFO* [TarMK flush thread: /opt/aem/publish/crx-quickstart/repository/segmentstore] org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK GC: Cleaned up file /opt/aem/publish/crx-quickstart/repository/segmentstore/data00175m.tar

When these start appearing, it runs through all of the .tar files and brings the system to a halt.  You cannot install any packages, replicate information, save a change to the repository in any way - which is an issue because it can take up to an hour to finish.  Is this the intended behavior for this?  Is there some configuration setting to change this?

1 Accepted Solution

Avatar

Correct answer by
Level 8

Just to update, since installing the Hotfix released today, this is no longer an issue.

View solution in original post

5 Replies

Avatar

Level 8

Just to follow up, the part that makes this take a long time is the following:

10.07.2014 15:55:48.666 *WARN* [TarMK flush thread: /opt/aem/publish/crx-quickstart/repository/segmentstore] org.apache.jackrabbit.oak.plugins.segment.file.TarReader Could not read file /opt/aem/publish/crx-quickstart/repository/segmentstore/data00135b.tar, skipping... java.io.FileNotFoundException: /opt/aem/publish/crx-quickstart/repository/segmentstore/data00135b.tar (No such file or directory) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241) at org.apache.jackrabbit.oak.plugins.segment.file.TarReader.openFirstFileWithValidIndex(TarReader.java:185) at org.apache.jackrabbit.oak.plugins.segment.file.TarReader.cleanup(TarReader.java:645) at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.flush(FileStore.java:364) at org.apache.jackrabbit.oak.plugins.segment.file.FileStore$1.run(FileStore.java:236) at java.lang.Thread.run(Thread.java:745) 10.07.2014 15:55:48.666 *WARN* [TarMK flush thread: /opt/aem/publish/crx-quickstart/repository/segmentstore] org.apache.jackrabbit.oak.plugins.segment.file.TarReader Failed to open cleaned up tar file /opt/aem/publish/crx-quickstart/repository/segmentstore/data00135a.tar

 

It then fails with the following, and i can begin using the repository again:

10.07.2014 15:59:51.473 *WARN* [10.200.1.47 [1405021996424] POST /bin/receive HTTP/1.1] org.apache.jackrabbit.oak.jcr.session.RefreshStrategy This session has been idle for 6 minutes and might be out of date. Consider using a fresh session or explicitly refresh the session. java.lang.Exception: The session was created here: at org.apache.jackrabbit.oak.jcr.session.RefreshStrategy$LogOnce.<init>(RefreshStrategy.java:124) at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl.createRefreshStrategy(RepositoryImpl.java:423) at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl.login(RepositoryImpl.java:253) at com.adobe.granite.repository.impl.CRX3RepositoryImpl.login(CRX3RepositoryImpl.java:87) at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl.login(RepositoryImpl.java:195) at org.apache.sling.jcr.base.AbstractSlingRepository2.login(AbstractSlingRepository2.java:297) at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProviderFactory.getResourceProviderInternal(JcrResourceProviderFactory.java:289) at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProviderFactory.getResourceProvider(JcrResourceProviderFactory.java:201) at org.apache.sling.resourceresolver.impl.tree.ResourceProviderFactoryHandler.login(ResourceProviderFactoryHandler.java:164) at org.apache.sling.resourceresolver.impl.tree.RootResourceProviderEntry.loginToRequiredFactories(RootResourceProviderEntry.java:95) at org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl.getResourceResolverInternal(CommonResourceResolverFactoryImpl.java:109) at org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl.getResourceResolver(CommonResourceResolverFactoryImpl.java:90) at org.apache.sling.resourceresolver.impl.ResourceResolverFactoryImpl.getResourceResolver(ResourceResolverFactoryImpl.java:93) at org.apache.sling.auth.core.impl.SlingAuthenticator.getResolver(SlingAuthenticator.java:760) at org.apache.sling.auth.core.impl.SlingAuthenticator.doHandleSecurity(SlingAuthenticator.java:483) at org.apache.sling.auth.core.impl.SlingAuthenticator.handleSecurity(SlingAuthenticator.java:438) at org.apache.sling.engine.impl.SlingHttpContext.handleSecurity(SlingHttpContext.java:121) at org.apache.felix.http.base.internal.context.ServletContextImpl.handleSecurity(ServletContextImpl.java:335) at org.apache.felix.http.base.internal.handler.ServletHandler.doHandle(ServletHandler.java:337) at org.apache.felix.http.base.internal.handler.ServletHandler.handle(ServletHandler.java:300) at org.apache.felix.http.base.internal.dispatch.ServletPipeline.handle(ServletPipeline.java:93) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:50) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.i18n.impl.I18NFilter.doFilter(I18NFilter.java:128) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.felix.http.sslfilter.internal.SslFilter.doFilter(SslFilter.java:55) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at com.adobe.granite.license.impl.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:298) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.security.impl.ReferrerFilter.doFilter(ReferrerFilter.java:290) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.featureflags.impl.FeatureManager.doFilter(FeatureManager.java:115) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.sling.engine.impl.log.RequestLoggerFilter.doFilter(RequestLoggerFilter.java:75) at org.apache.felix.http.base.internal.handler.FilterHandler.doHandle(FilterHandler.java:108) at org.apache.felix.http.base.internal.handler.FilterHandler.handle(FilterHandler.java:80) at org.apache.felix.http.base.internal.dispatch.InvocationFilterChain.doFilter(InvocationFilterChain.java:46) at org.apache.felix.http.base.internal.dispatch.HttpFilterChain.doFilter(HttpFilterChain.java:31) at org.apache.felix.http.base.internal.dispatch.FilterPipeline.dispatch(FilterPipeline.java:76) at org.apache.felix.http.base.internal.dispatch.Dispatcher.dispatch(Dispatcher.java:49) at org.apache.felix.http.base.internal.DispatcherServlet.service(DispatcherServlet.java:67) at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745)

Avatar

Level 10

leeasling wrote...

I've noticed when replicating large amounts of data, or installing large packages, we see the following in our error logs:

10.07.2014 08:18:11.045 *INFO* [TarMK flush thread: /opt/aem/publish/crx-quickstart/repository/segmentstore] org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK GC: Cleaned up file /opt/aem/publish/crx-quickstart/repository/segmentstore/data00175m.tar

When these start appearing, it runs through all of the .tar files and brings the system to a halt.  You cannot install any packages, replicate information, save a change to the repository in any way - which is an issue because it can take up to an hour to finish.  Is this the intended behavior for this?  Is there some configuration setting to change this?

 

Are you sure you are using aem6 GA copy?  If so please file a daycare & share your package.

Avatar

Level 8

Sham HC wrote...

leeasling wrote...

I've noticed when replicating large amounts of data, or installing large packages, we see the following in our error logs:

10.07.2014 08:18:11.045 *INFO* [TarMK flush thread: /opt/aem/publish/crx-quickstart/repository/segmentstore] org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK GC: Cleaned up file /opt/aem/publish/crx-quickstart/repository/segmentstore/data00175m.tar

When these start appearing, it runs through all of the .tar files and brings the system to a halt.  You cannot install any packages, replicate information, save a change to the repository in any way - which is an issue because it can take up to an hour to finish.  Is this the intended behavior for this?  Is there some configuration setting to change this?

 

Are you sure you are using aem6 GA copy?  If so please file a daycare & share your package.

 

It was a copy that was requested through support after it went live, so i assume so.  Is there a way to tell in the console?

It's not possible to share the packages as they amount to around 120GB.

Avatar

Correct answer by
Level 8

Just to update, since installing the Hotfix released today, this is no longer an issue.

Avatar

Level 3

Hi ,

I have same issue , which hotfix was applied to fix this issue?