Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
BedrockMission!

Learn More

View all

Sign in to view all badges

High load due to ContentMirrorStoreStrategy?

Avatar

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile
Jdruwe
Level 4

01-08-2017

We are experiencing extremely high load on our AEM Author (6.1, Oak 1.2.7) Our logs get filled by the following ContentMirrorStoreStrategy messages. I am not sure what they mean and if they are the cause of the load, any help would be greatly appreciated. NOTE: I only get this log for our 'Publisher A'

During this phenomenon we are unable to rollout any pages to their livecopy.

Extra info about our setup:

  • 1 author
  • 3 publishers (A, B, C)
  • 8+ sites

01.08.2017 09:53:54.630 *WARN* [10.1.106.97 [1501581233267] GET /libs/wcm/core/content/pageinfo.json HTTP/1.1] org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy Traversed 70000 nodes (678 index entries) using index event.job.topic with

filter Filter(query=select [jcr:path], [jcr:score], * from [slingevent:Job] as a where [event.job.topic] = 'com/day/cq/replication/job/publisha' and [slingevent:finishedState] is null order by [slingevent:created] /* xpath: //element(*,slingevent:Job)[@event.job.topic = 'com/day/cq/replication/job/publisha'

and not(@slingevent:finishedState)] order by @slingevent:created ascending */, path=*, property=[slingevent:finishedState=[is null], event.job.topic=[com/day/cq/replication/job/publisha]])

Greetings

Jeroen

Replies

Avatar

Avatar
Level 1
vamshikega
Level 1

Likes

0 likes

Total Posts

3 posts

Correct Reply

0 solutions
View profile

Avatar
Level 1
vamshikega
Level 1

Likes

0 likes

Total Posts

3 posts

Correct Reply

0 solutions
View profile
vamshikega
Level 1

01-08-2017

Hi Jereon,

Refer this thread http://https://forums.adobe.com/thread/2323570

Thanks

Vamshi

Avatar

Avatar
Validate 25
Level 10
smacdonald2008
Level 10

Likes

1,406 likes

Total Posts

12,671 posts

Correct Reply

2,278 solutions
Top badges earned
Validate 25
Validate 10
Validate 1
Give back 900
Give back 600
View profile

Avatar
Validate 25
Level 10
smacdonald2008
Level 10

Likes

1,406 likes

Total Posts

12,671 posts

Correct Reply

2,278 solutions
Top badges earned
Validate 25
Validate 10
Validate 1
Give back 900
Give back 600
View profile
smacdonald2008
Level 10

01-08-2017

We did a weinar with top community members that discussed (and with performance in mind)--

This session of Ask the AEM Community Experts will share large scale architectures from the author's experiences with various companies like Cisco and Symantec and compare and contrast the architecture across: Infrastructure Architecture Scaling

Ecommerce integrations and migration approach from legacy into Adobe Experience Manager, Digital Marketing Cloud Integrations such as personalization, analytics, and DMP.

This may help you -- Scott's Digital Community: Comparative Architecture Analysis of large scale Experience Manager Insta...

Avatar

Avatar
Coach
Employee
Jörg_Hoh
Employee

Likes

1,081 likes

Total Posts

3,121 posts

Correct Reply

1,061 solutions
Top badges earned
Coach
Give back 600
Ignite 5
Ignite 3
Ignite 1
View profile

Avatar
Coach
Employee
Jörg_Hoh
Employee

Likes

1,081 likes

Total Posts

3,121 posts

Correct Reply

1,061 solutions
Top badges earned
Coach
Give back 600
Ignite 5
Ignite 3
Ignite 1
View profile
Jörg_Hoh
Employee

01-08-2017

Hi,

this is caused by an inperformant index; in your case I would

1) update to a later version of Oak (which comes with some improvements also in terms of performance)

2) raise a Daycare ticket and ask if there are recommendations how you can tune your indexes.

Jörg

Avatar

Avatar
Give Back 50
Employee
Feike_Visser1
Employee

Likes

247 likes

Total Posts

970 posts

Correct Reply

221 solutions
Top badges earned
Give Back 50
Give Back 5
Give Back 3
Give Back 25
Give Back 10
View profile

Avatar
Give Back 50
Employee
Feike_Visser1
Employee

Likes

247 likes

Total Posts

970 posts

Correct Reply

221 solutions
Top badges earned
Give Back 50
Give Back 5
Give Back 3
Give Back 25
Give Back 10
View profile
Feike_Visser1
Employee

01-08-2017

Can you install CFP1 or later and retest?

Avatar

Avatar
Boost 5
Employee
vmehrotr
Employee

Likes

19 likes

Total Posts

43 posts

Correct Reply

5 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile

Avatar
Boost 5
Employee
vmehrotr
Employee

Likes

19 likes

Total Posts

43 posts

Correct Reply

5 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile
vmehrotr
Employee

01-08-2017

Use Lucene Sling Event index instead of using event.job.topic property index. Event.job.topic property index is very slow.

It seems that you still have event.job.topic property index in place. Your query is traversing even when it uses event.job.topic index since it has way too many entries. Property indexes are not very performant if you are indexing a lot of nodes.

When you have slingeventJob lucene index, then you would need to remove that property slingevent:job from nodetype index.

You would have slingevent:job as property in declared nodetype index. Also, you would have event.job.topic property index in place.

The recommendation from engineering has been to convert it to lucene based index definition.

So, you could have a lucene based index for it which should help the cause.

<slingeventJob

            jcr:primaryType="oak:QueryIndexDefinition"

            async="async"

            compatVersion="{Long}2"

            name="slingevent:Job"

            reindex="{Boolean}true"

            indexPath="/oak:index/slingeventJob"

            includedPaths="[/var/eventing]"

            type="lucene">

        <indexRules jcr:primaryType="nt:unstructured">

            <slingevent:Job jcr:primaryType="nt:unstructured">

                <properties jcr:primaryType="nt:unstructured">

                    <eventJobTopic

                            jcr:primaryType="nt:unstructured"

                            name="event.job.topic"

                            ordered="{Boolean}false"

                            propertyIndex="{Boolean}true"

                            type="String"/>

                    <slingeventCreated

                            jcr:primaryType="nt:unstructured"

                            name="slingevent:created"

                            ordered="{Boolean}true"

                            propertyIndex="{Boolean}true"

                            type="Date"/>

                    <finishedState

                            jcr:primaryType="nt:unstructured"

                            name="slingevent:finishedState"

                            ordered="{Boolean}false"

                            propertyIndex="{Boolean}true"

                            nullCheckEnabled="{Boolean}true"

                            type="String"/>

                </properties>

            </slingevent:Job>

        </indexRules>

    </slingeventJob>

and once this gets reindexed

Then you can add a property to the ootb property index event.job.topic by performing below:

Add below property to event.job.topic index so that it doesn't get picked

b) entryCount

Long

99999999

Further, also remove the slingevent:Job from the declaringNodetypes property in nodetype index

"nodetype": {

    "reindexCount": 1,

    "reindex": false,

    "nodeTypeListDefined": true,

    "type": "property",

    "propertyNames": [

      "jcr:primaryType",

      "jcr:mixinTypes"

    ],

    "jcr:primaryType": "oak:QueryIndexDefinition",

    "declaringNodeTypes": [

      "cq:PollConfigFolder",

      "sling:MessageEntry",

      "cq:ExporterConfigFolder",

      "cq:BlueprintSyncConfig",

      "cq:ClientLibraryFolder",

      "granite:Task",

      "cq:Template",

      "slingevent:Job",

      "granite:CloudsettingsConfigType",

      "oak:QueryIndexDefinition",

      "rep:User",

      "rep:Authorizable",

      "cq:PollConfig",

      "sling:bgJobData",

      "cq:ExporterConfig",

      "sling:VanityPath",

      "sling:chunks",

      "mix:language",

      "cq:Component",

      "cq:Console",

      "slingevent:TimedEvent",

      "cq:ContentSyncConfig"

    ]

Avatar

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile
Jdruwe
Level 4

01-08-2017

I am going to try this, really hoping this will work vmehrotr​, you said 'and once this gets reindexed', how do I know it is reindexed?

Avatar

Avatar
Boost 5
Employee
vmehrotr
Employee

Likes

19 likes

Total Posts

43 posts

Correct Reply

5 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile

Avatar
Boost 5
Employee
vmehrotr
Employee

Likes

19 likes

Total Posts

43 posts

Correct Reply

5 solutions
Top badges earned
Boost 5
Boost 3
Boost 10
Boost 1
Applaud 5
View profile
vmehrotr
Employee

01-08-2017

Once you create the new index, you would see message like below in logs.

*INFO* [pool-9-thread-1] org.apache.jackrabbit.oak.plugins.index.IndexUpdate

Reindexing would be performed for following indexes [/oak:index/slingeventJob]

..

*INFO* [pool-9-thread-1] org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate

Reindexing completed for indexes: [/oak:index/slingeventJob] in x.xxxx h

Avatar

Avatar
Coach
Employee
Jörg_Hoh
Employee

Likes

1,081 likes

Total Posts

3,121 posts

Correct Reply

1,061 solutions
Top badges earned
Coach
Give back 600
Ignite 5
Ignite 3
Ignite 1
View profile

Avatar
Coach
Employee
Jörg_Hoh
Employee

Likes

1,081 likes

Total Posts

3,121 posts

Correct Reply

1,061 solutions
Top badges earned
Coach
Give back 600
Ignite 5
Ignite 3
Ignite 1
View profile
Jörg_Hoh
Employee

02-08-2017

Hi,

please be advised, that this process can take some hours (maybe even days)! So this is not an operation you should do without testing.

I also advised you to ask Daycare support regarding this, because this index definition is not the only one which you should replace. Support has a number of recommendations regarding it.

Jörg

Avatar

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile

Avatar
Validate 10
Level 4
Jdruwe
Level 4

Likes

29 likes

Total Posts

113 posts

Correct Reply

7 solutions
Top badges earned
Validate 10
Validate 1
Boost 5
Boost 3
Boost 25
View profile
Jdruwe
Level 4

02-08-2017

I installed Cumulative Fix Pack 10 pack but I am now seeing the following error messages in the error.log:

02.08.2017 13:48:53.146 *ERROR* [IdMapService-initializer] org.apache.sling.discovery.commons.providers.spi.base.IdMapService initializer: could not init due to java.lang.NullPointerException

java.lang.NullPointerException: null

at org.apache.sling.discovery.commons.providers.spi.base.IdMapService.init(IdMapService.java:187)

at org.apache.sling.discovery.commons.providers.spi.base.IdMapService.access$000(IdMapService.java:59)

at org.apache.sling.discovery.commons.providers.spi.base.IdMapService$1.check(IdMapService.java:108)

at org.apache.sling.discovery.commons.providers.spi.base.AbstractServiceWithBackgroundCheck$BackgroundCheckRunnable.run(AbstractServiceWithBackgroundCheck.java:77)

at java.lang.Thread.run(Thread.java:745)

I found this ticket: [SLING-5592] rapid IdMapService activation/deactivation leaves background thread in limbo - ASF JIRA , it seems to be fixed in version 1.0.12. AEM 6.1 seems to be using 1.0.4:

Screen Shot 2017-08-02 at 13.50.52.png

Is there a way to fix this?