One of our customer has migrated a lot of content (assets and pages)... they have about 1.5 million assets and about 4 million site pages. Part of the sites are generated by XML Add-on. Problem:With this much load obviously Author (AEM 6.4) is slow for authoring. And sometimes crashes. Solutions:The server where AEM author is running has been upgraded to have a lot of resources (94 core CPU and 384GB memory).. this maybe not be best solution but did not help a lot.Going through all the AEM systems configuration (workflows, any orphan process, etc.) Using ACS Commons' Bulk Workflow tool.Review an Elastic search index configured that runs weeklyFollowing offline-compaction and other suggested per the AEM 6.x Maintenance Guide Usually what can be done with repository of such huge sizes?

Thanks for chiming in Jorg.The answer to most questions is "XML AddOn"From a handling point-of-view, does it make sense to have them all in AEM? [ Authors are copying the dita content into Assets from external source and after editing the content, generating the output. The generated output goes under "/sites" which is then published to Publisher. ]How are your authors interacting them? Are they at all? [ If there are any updates.modifications to original dita content, they can go back to the assets and re-generate the output/sites-pages. ]Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)? [ Manual ]So this process (along with some workflows that run by the xml add-on) is creating a lot of nodes/content on Author and is bloating it up. Maybe it's more of a process that needs to be updated than any issues on AEM

It's really a very huge data. I have few things in mind which could be help full?External data store: When dealing with large number of binaries, it is recommended that an external data store be used instead of the default node stores in order to maximize performance Configuring node stores and data stores in AEM 6 Query Optimazation: Use http://localhost:4502/libs/granite/operations/content/diagnosistools/requestPerformance.html to anaylize what are the most time taking queries which might cause of aem performance.Indexing: Check error log and search "consider creating an index or changing the query", which can give you idea if any missing index. Oak Queries and Indexing

surenk

Level 4

Solved

Lots of content - AEM Author performance issues

Forum|Forum|6 years ago
October 21, 2019
10 replies
8596 views

One of our customer has migrated a lot of content (assets and pages)... they have about 1.5 million assets and about 4 million site pages. Part of the sites are generated by XML Add-on.

Problem:

With this much load obviously Author (AEM 6.4) is slow for authoring. And sometimes crashes.

Solutions:

The server where AEM author is running has been upgraded to have a lot of resources (94 core CPU and 384GB memory).. this maybe not be best solution but did not help a lot.
Going through all the AEM systems configuration (workflows, any orphan process, etc.)
- Using ACS Commons' Bulk Workflow tool.
Review an Elastic search index configured that runs weekly
Following offline-compaction and other suggested per the AEM 6.x Maintenance Guide

Usually what can be done with repository of such huge sizes?

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by surenk

Thanks for chiming in Jorg.

The answer to most questions is "XML AddOn"

From a handling point-of-view, does it make sense to have them all in AEM? [ Authors are copying the dita content into Assets from external source and after editing the content, generating the output. The generated output goes under "/sites" which is then published to Publisher. ]
How are your authors interacting them? Are they at all? [ If there are any updates.modifications to original dita content, they can go back to the assets and re-generate the output/sites-pages. ]
Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)? [ Manual ]

So this process (along with some workflows that run by the xml add-on) is creating a lot of nodes/content on Author and is bloating it up. Maybe it's more of a process that needs to be updated than any issues on AEM

BrijeshYadav

Level 5

It's really a very huge data.

I have few things in mind which could be help full?

External data store: When dealing with large number of binaries, it is recommended that an external data store be used instead of the default node stores in order to maximize performance Configuring node stores and data stores in AEM 6
Query Optimazation: Use http://localhost:4502/libs/granite/operations/content/diagnosistools/requestPerformance.html to anaylize what are the most time taking queries which might cause of aem performance.
Indexing: Check error log and search "consider creating an index or changing the query", which can give you idea if any missing index. Oak Queries and Indexing

R

rachnamehta

Level 3

Hi,

I think your AEM instance might not be tuned properly. I have written short summary of "Sling Memory Deep Dive" here https://medium.com/the-telegraph-engineering/four-highlights-from-adaptto-2018-3781782a6b7a and a detailed session video available here adaptTo() 2018 I found it very interesting in terms of memory usage, you may want to check it.

Running compaction will obviously help and 6.4 does online compaction already so make sure it's happening.

Creating indexes based on your queries will help too.

S

Santhi_Swaroop

Level 2

Try to shard the content/sites to two different authors and publish to same publish instances so that you can minimize the load by 50%

surenkAuthor

Level 4

Forgot to mention.. the data is on external AWS S3 drive and referred by AEM.

sunjot16

Adobe Employee

Perform Revision Cleanups, Version Purge, Workflow Purge, etc.

The following could also help:

Monitoring and Maintaining Your AEM instance

AEM 6.x Maintenance Guide

joerghoh

Adobe Employee

Hi,

I wonder what value comes with lots of pages, which are auto-generated and probably also auto-maintained in AEM. From a handling point-of-view, does it make sense to have them all in AEM? How are your authors interacting them? Are they at all? Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)?

Same with assets.

If you cannot answer these questions, I would try to find a way to maintain all this content outside of AEM. If you manage this content only by automation, what's the value of having them in AEM?

surenkAuthorAccepted solution

Level 4

Thanks for chiming in Jorg.

The answer to most questions is "XML AddOn"

From a handling point-of-view, does it make sense to have them all in AEM? [ Authors are copying the dita content into Assets from external source and after editing the content, generating the output. The generated output goes under "/sites" which is then published to Publisher. ]
How are your authors interacting them? Are they at all? [ If there are any updates.modifications to original dita content, they can go back to the assets and re-generate the output/sites-pages. ]
Or are just automatic processes working with them all the time (regenerating, publishing, deactivating)? [ Manual ]

So this process (along with some workflows that run by the xml add-on) is creating a lot of nodes/content on Author and is bloating it up. Maybe it's more of a process that needs to be updated than any issues on AEM

joerghoh

Adobe Employee

Ah, XML addon. But unfortunately I cannot help you with it.

In any way, I would get in contact with Adobe support and tell them about your case. And ask them for suggestions to help you here.

surenkAuthor

Level 4

We are working with them as well. There are some known issues!!

joerghoh

Adobe Employee

Hi Surenk,

would you mind to post a small summary about how you were able to solve your issue after you completed the work with support? I am quite sure others are interested as well.

Thanks,

Jörg

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded