AEM Author prod instance slows down too quick

[salvadorm142499]

Hi all,

The production author AEM instance of my client slows down too quick. After reading the logs I have found a couple of heavy reasons:

1. Opened sessions for a very long time and I don't know the reason: org.apache.jackrabbit.oak.jcr.session.RefreshStrategy This session has been idle for 4 minutes and might be out of date. Consider using a fresh session or explicitly refresh the session.
   • To get a ResourceResolver within an OSGi Service, we use:
     • @Reference
       protected ResourceResolverFactory resourceResolverFactory;
     • resourceResolverFactory.getServiceResourceResolver(ImmutableMap.of(ResourceResolverFactory.SUBSERVICE, "myService"));
     • Add myService user to user.mapping within the right run mode config folder within org.apache.sling.serviceusermapping.impl.ServiceUserMapperImpl.xml
   • Always closing the resourceResolver on the finally block.
2. AEM OOTB requests for page/asset information are too slow: 3895523ms 10/Dec/2015:13:49:30 +0100 500 GET /libs/wcm/core/content/reference.json?path=/content/some/path&_charset_=UTF-8&_dc=1449751180475 text/html

Some information about the intance usage:

• Multilanguage site: No blueprint has been used.
  •  Master (master) -> Public Master copy (live copy of master) -> com (live copy of public master) & countries (more than 30 countries, each one another live copy of public master copy)
• More than 50 concurrent users
• No author clustering

After restarting the instance I have tried: /libs/wcm/core/content/reference.json?path=/content/to/some/path and returned no assets and the request took 1.6 mins to be completed. How could I set a timeout for any request?

Many thanks in advance,

Salva.

[smacdonald2008]

If you are querying and its taking that long - looks like an index issue.  Look at this AEM topic:

https://docs.adobe.com/docs/en/aem/6-0/deploy/upgrade/queries-and-indexing.html

Without an indexc -- AEM searches the entire JCR - this explains why your search took so long. 

[smacdonald2008]

Also - ensure that you have the hotfixes installed. Have you installed the hotfixes? 

[Sham_HC]

follow https://helpx.adobe.com/experience-manager/kb/performance-tuning-tips.html

[Paul_McMahon · Accepted solution]

50 concurrent users is close to the top end of the performance profile for a single authoring instance. Which version of the software are you on? What sort of hardware are you running on - what number of CPUs and how much memory? How is your storage connected - do you have a fully optimized IO solution? 

Have you considered that you may be at the point that you need to implement a cluster or off loading strategy? Do you have a performance testing environment. Are you able to duplicate these scenarios under load in you performance testing environment? 

[salvadorm142499]

Hi all,

Thanks for all your replies.

We are running AEM 6.0 SP3 with hotfixes installed up to date. We have also already carried out performance tunings for indexing at: https://helpx.adobe.com/experience-manager/kb/performance-tuning-tips.html

About the machine it is running on, it is a 16 Gb Ram machine (AEM is running on 12Gb), 8 cores and enough disk space. We have a pre-prod environment but the load and the amount of content is not comparable to prod environment (Unfortunately, I am not in charge of these environments).

Aside of indexes (I will further investigate it), any other reason? 

As the content editorial team does not use too much versioning, we have also applied a strict versioning strategy (less number of versions and expire sooner) but, I found this error line too many times:

10.12.2015 02:00:40.538 *ERROR* [pool-7-thread-19-Maintenance Queue(com/adobe/granite/maintenance/job/VersionPurgeTask)] com.day.cq.wcm.core.impl.VersionManagerImpl Unable to purge version 1.0 for /content/editorial-master-testing-it/en/carworlds/leon-sc/specifications/engines/petrol/engine-config-12-tsi-110hp/jcr:content : OakIntegrity0001: Unable to delete referenced node

Could it also be related with indexing?

[salvadorm142499]

orotas wrote...

50 concurrent users is close to the top end of the performance profile for a single authoring instance. Which version of the software are you on? What sort of hardware are you running on - what number of CPUs and how much memory? How is your storage connected - do you have a fully optimized IO solution? 

Have you considered that you may be at the point that you need to implement a cluster or off loading strategy? Do you have a performance testing environment. Are you able to duplicate these scenarios under load in you performance testing environment? 

Hi orotas,

I have answered on my last post providing you some more information about the instance.

[salvadorm142499]

Sham HC wrote...

follow https://helpx.adobe.com/experience-manager/kb/performance-tuning-tips.html

Hi Sham,

Yes, we have applied tuning tips for AEM 6. Some more information is on my last post.

Cheers,

Salva.

[salvadorm142499]

smacdonald2008 wrote...

Also - ensure that you have the hotfixes installed. Have you installed the hotfixes? 

Hi smacdonald,

Yes, all hot fixes and Service Packs have been installed. You could find some more information on my last post. Any other reason a part of indexes? I will do some research about the link you suggested.

Many thanks,

Salva.

[Mohan_R2]

Hi Selva,

We facing the same issue with AEM 6.1 (no service packs installed). Can you please let me know, how the issue resolved?

Regards,

Mohan.