Open sessions - JMX

Veena_Vikram

MVP

03-08-2018

Hi All

    I am trying to figure out the code which has a session leak. I am following Adobe's documentation.Check and analyze if JCR session leaks in your AEM instance

But I have few models where a resourceResolver is injected using

@SlingObject

    private ResourceResolver resolver;

    I can see few class reference in JMX which has these classes mentioned in the stack trace. How will we handle these resolvers? Will these be closed automatically ?

TIA

Veena

Accepted Solutions (1)

Accepted Solutions (1)

Jörg_Hoh

Employee

07-08-2018

I just checked [1] and I haven't found a way that the ResourceTypeResolver (which seems to leak in your case) is actually leaking. It is always closed (line 665ff).

I cannot explain the fact that a lot of these session are actually open (it would be good if you could verify that by a heapdump), and that it's not a problem of the garbage collection.

[1] sling-org-apache-sling-resourceresolver/ResourceResolverControl.java at org.apache.sling.resourceres...

Answers (14)

Answers (14)

Jörg_Hoh

Employee

05-09-2018

Below that (and not contained in the screenshot) there is a field "stacktrace", which should be filled with the stacktrace. This stacktraces are only created when more than 1000 sessions are opened (default), so you should check the youngest sessions, they should contain these stacks.

Veena_Vikram

MVP

06-08-2018

Thanks Jörg Hoh​ .

A little insight to the situation right now ,The situation is bit bad with our prod servers right now. Since business reported slowness , we went back to Adobe and they informed us to check JMX for open sessions and fix the code. The sessions and resourceResolvers in our code was already taken care. I checked JMX and yesterday alone there was >11000 Session objects (note that this is the count of sessions opened in last 24 hours ) tracked in JMX. I read that anything above 500 is bad. I checked first 50 manually and all showed the above stack trace. Then I checked randomly from all these 11000 and few didn't had a stack trace and the rest were all these and few from one old project which was still in prod (this will be deleted).

Jörg_Hoh

Employee

06-08-2018

A little bit of background: A resource resolver internally holds 2 session! One "real" session through which all actions are performed, and this session is running with user privileges. And an additional one with elevated privileges which is used for resolving privileges and such (which is typically not always directly accesible by the user session).

I remember an issue with this additional session but I don't recall any details.

Also it could be temporary glitch with Garbage Collection and lazily unregistering MBeans, I saw reports on the sling JIRA as well. Maybe you want to check if all (well, kind of) sessions have the same threaddump for opening and then report it to Adobe support.

Veena_Vikram

MVP

06-08-2018

Thanks Jörg Hoh​ . As you can see from the method signature, we are just passing a Page and String as arguments. But the real culprit is the method isResoureType() in Resource ("The Adobe AEM Quickstart and Web Application.") . If you see the method details in API, it is saying that

So if we go by above does that mean this method is opening a resourceResolver and is never closed ?

Jörg_Hoh

Employee

06-08-2018

What version of AEM are you using?

This observation seems somehow related to [1], although this did not mention a leak of JCR sessions. IIRC there was another issue in that area which actually dealt with a leak loosing the ResourceTypeResolver RR, but I don't find it rightaway. How do you obtain the the resourceResolver you feed into this "isResourceType" call? I am asking because of [2].

[1] [SLING-3285] ResourceResolver.isResourceType() creates admin session for every call with super type,...

[2] [SLING-7591] Session leak when a ResourceResolver is created from a Session then gets cloned - ASF J...

Veena_Vikram

MVP

06-08-2018

Thanks Arun , Jorg . The sessions and resourceResolver I opened is closed. But yet there are around 6500 open sessions in P1 which I am not sure what is causing. I tried manually opening each session detail given in JMX and tried to check the stack trace. Basically the stack trace points to me a class , where I have not opened any sessions implicitly, rather I can only see this line

1542958_pastedImage_0.png

So as per the stack trace page.getContentResource().isResourceType(resourceType) is one culprit in the chain whcih opened the session. Please see the stack trace below.

java.lang.Exception: The session was opened here:

at org.apache.jackrabbit.oak.jcr.session.SessionStats.<init>(SessionStats.java:84)

at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.<init>(SessionDelegate.java:150)

at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl$1.<init>(RepositoryImpl.java:301)

at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl.createSessionDelegate(RepositoryImpl.java:299)

at org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl.login(RepositoryImpl.java:281)

at com.adobe.granite.repository.impl.CRX3RepositoryImpl.login(CRX3RepositoryImpl.java:94)

at com.adobe.granite.repository.impl.SlingRepositoryImpl$2.run(SlingRepositoryImpl.java:112)

at com.adobe.granite.repository.impl.SlingRepositoryImpl$2.run(SlingRepositoryImpl.java:104)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)

at com.adobe.granite.repository.impl.SlingRepositoryImpl.createAdministrativeSession(SlingRepositoryImpl.java:104)

at org.apache.sling.jcr.base.AbstractSlingRepository2.loginAdministrative(AbstractSlingRepository2.java:396)

at org.apache.sling.jcr.resource.internal.helper.jcr.JcrProviderStateFactory.createProviderState(JcrProviderStateFactory.java:82)

at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProvider.authenticate(JcrResourceProvider.java:267)

at org.apache.sling.jcr.resource.internal.helper.jcr.JcrResourceProvider.authenticate(JcrResourceProvider.java:78)

at org.apache.sling.resourceresolver.impl.providers.stateful.ProviderManager.authenticate(ProviderManager.java:161)

at org.apache.sling.resourceresolver.impl.providers.stateful.ProviderManager.getOrCreateProvider(ProviderManager.java:87)

at org.apache.sling.resourceresolver.impl.providers.stateful.ProviderManager.authenticateAll(ProviderManager.java:129)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.createControl(ResourceResolverImpl.java:154)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.<init>(ResourceResolverImpl.java:116)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.<init>(ResourceResolverImpl.java:110)

at org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl.getResourceResolverInternal(CommonResourceResolverFactoryImpl.java:258)

at org.apache.sling.resourceresolver.impl.CommonResourceResolverFactoryImpl.getAdministrativeResourceResolver(CommonResourceResolverFactoryImpl.java:141)

at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.getResourceTypeResourceResolver(ResourceResolverControl.java:689)

at org.apache.sling.resourceresolver.impl.helper.ResourceResolverControl.getParentResourceType(ResourceResolverControl.java:713)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.getParentResourceType(ResourceResolverImpl.java:1304)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.getParentResourceType(ResourceResolverImpl.java:1293)

at org.apache.sling.resourceresolver.impl.ResourceResolverImpl.isResourceType(ResourceResolverImpl.java:1321)

at org.apache.sling.api.resource.AbstractResource.isResourceType(AbstractResource.java:123)

at com.<****>.aem.website.v1.core.utils.PageUtil.isResourceType(PageUtil.java:147)

Hope this will help more