Lots of content - AEM Author performance issues

[surenk]

One of our customer has migrated a lot of content (assets and pages)... they have about 1.5 million assets and about 4 million site pages. Part of the sites are generated by XML Add-on.

Problem:

With this much load obviously Author (AEM 6.4) is slow for authoring. And sometimes crashes.

Solutions:

• The server where AEM author is running has been upgraded to have a lot of resources (94 core CPU and 384GB memory).. this maybe not be best solution but did not help a lot.
• Going through all the AEM systems configuration (workflows, any orphan process, etc.)
  • Using ACS Commons' Bulk Workflow tool.
• Review an Elastic search index configured that runs weekly
• Following offline-compaction and other suggested per the AEM 6.x Maintenance Guide

Usually what can be done with repository of such huge sizes?

[BrijeshYadav]

It's really a very huge data.

I have few things in mind which could be help full?

1. External data store: When dealing with large number of binaries, it is recommended that an external data store be used instead of the default node stores in order to maximize performance  Configuring node stores and data stores in AEM 6
2. Query Optimazation: Use http://localhost:4502/libs/granite/operations/content/diagnosistools/requestPerformance.html  to anaylize what are the most time taking queries which might cause of aem performance.
3. Indexing: Check error log and search "consider creating an index or changing the query", which can give you idea if any missing index. Oak Queries and Indexing

[rachanam1547401]

Hi,

I think your AEM instance might not be tuned properly. I have written short summary of "Sling Memory Deep Dive" here https://medium.com/the-telegraph-engineering/four-highlights-from-adaptto-2018-3781782a6b7a   and a detailed session video available here adaptTo() 2018   I found it very interesting in terms of memory usage, you may want to check it.

Running compaction will obviously help and 6.4 does online compaction already so make sure it's happening.

Creating indexes based on your queries will help too.

[Santhi_Swaroop]

Try to shard the content/sites to two different authors and publish to same publish instances so that you can minimize the load by 50%

[surenk]

Forgot to mention.. the data is on external AWS S3 drive and referred by AEM.

[sunjot16]

Perform Revision Cleanups, Version Purge, Workflow Purge, etc.

The following could also help:

Monitoring and Maintaining Your AEM instance

AEM 6.x Maintenance Guide

[Jörg_Hoh]

Hi,

I wonder what value comes with lots of pages, which are auto-generated and probably also auto-maintained in AEM. From a handling point-of-view, does it make sense to have them all in AEM? How are your authors interacting them? Are they at all? Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)?

Same with assets.

If you cannot answer these questions, I would try to find a way to maintain all this content outside of AEM. If you manage this content only by automation, what's the value of having them in AEM?

[surenk · Accepted solution]

Thanks for chiming in Jorg.

The answer to most questions is "XML AddOn"

1. From a handling point-of-view, does it make sense to have them all in AEM? [ Authors are copying the dita content into Assets from external source and after editing the content, generating the output. The generated output goes under "/sites" which is then published to Publisher. ]
2. How are your authors interacting them? Are they at all? [ If there are any updates.modifications to original dita content, they can go back to the assets and re-generate the output/sites-pages. ]
3. Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)? [ Manual ]

So this process (along with some workflows that run by the xml add-on) is creating a lot of nodes/content on Author and is bloating it up. Maybe it's more of a process that needs to be updated than any issues on AEM

[Jörg_Hoh]

Ah, XML addon. But unfortunately I cannot help you with it.

In any way, I would get in contact with Adobe support and tell them about your case. And ask them for suggestions to help you here.

[surenk]

We are working with them as well. There are some known issues!!

[Jörg_Hoh]

Hi Surenk,

would you mind to post a small summary about how you were able to solve your issue after you completed the work with support? I am quite sure others are interested as well.

Thanks,

Jörg