Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
BedrockMission!

Learn More

View all

Sign in to view all badges

AEM publish servers Taking too long to stop

Avatar

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile
AnkurAhlawat
Level 6

26-07-2016

AEM publish servers are taking too long to stop, no error is getting printed in error log too. Some time it is taking upto 150 minutes + and then we need to kill the process. But have fear that it might corrupt the repository. Any help or pointers are appreciated.

Replies

Avatar

Avatar
Validate 1
Employee
Opkar_Gill
Employee

Likes

138 likes

Total Posts

952 posts

Correct Reply

280 solutions
Top badges earned
Validate 1
Give Back 50
Give Back 5
Give Back 3
Give Back 25
View profile

Avatar
Validate 1
Employee
Opkar_Gill
Employee

Likes

138 likes

Total Posts

952 posts

Correct Reply

280 solutions
Top badges earned
Validate 1
Give Back 50
Give Back 5
Give Back 3
Give Back 25
View profile
Opkar_Gill
Employee

26-07-2016

Hi,

it would help if you told us:

  1. What version of AEM this is
  2. Any configurations you have done i.e. external file datastore, "nocamplecontent"
  3. Any hot fixes or service packs
  4. Has this issue only started happening recently, have there been any changes to the repository: Code deployment, SP or hot fix

Regards,

Opkar

Avatar

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile
AnkurAhlawat
Level 6

26-07-2016

I am using AEM 6.1, with basic hotfixes. We are not using any external file store. We are using cold and standy architecture.

This issue has started from past 1 month only, and we have migrated from CQ5.6 to AEM 6.1 3 months back. For first two months we haven't faced this issue.

And this issue is not consistent on a specific server sometimes it happens on server 2 some time server 3. We were planning to put SP1 but we have put it on hold because of this.

Avatar

Avatar
Validate 50
Level 5
askdctm
Level 5

Likes

48 likes

Total Posts

476 posts

Correct Reply

7 solutions
Top badges earned
Validate 50
Validate 25
Validate 10
Validate 1
Boost 5
View profile

Avatar
Validate 50
Level 5
askdctm
Level 5

Likes

48 likes

Total Posts

476 posts

Correct Reply

7 solutions
Top badges earned
Validate 50
Validate 25
Validate 10
Validate 1
Boost 5
View profile
askdctm
Level 5

27-07-2016

Hi Ankur,

Probably taking thread/heap dumps during that time frame will give some pointers.

Avatar

Avatar
Validate 1
Employee
Opkar_Gill
Employee

Likes

138 likes

Total Posts

952 posts

Correct Reply

280 solutions
Top badges earned
Validate 1
Give Back 50
Give Back 5
Give Back 3
Give Back 25
View profile

Avatar
Validate 1
Employee
Opkar_Gill
Employee

Likes

138 likes

Total Posts

952 posts

Correct Reply

280 solutions
Top badges earned
Validate 1
Give Back 50
Give Back 5
Give Back 3
Give Back 25
View profile
Opkar_Gill
Employee

27-07-2016

Hi,

You should raise a daycare ticket about this. Thread dumps will help to identify what is blocking shutdown in your instance. Did you do a code deployment just before this issue started?

Regards,

Opkar

Avatar

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile

Avatar
Ignite 5
Level 6
AnkurAhlawat
Level 6

Likes

111 likes

Total Posts

165 posts

Correct Reply

27 solutions
Top badges earned
Ignite 5
Ignite 3
Ignite 20
Ignite 10
Ignite 1
View profile
AnkurAhlawat
Level 6

27-07-2016

We have already raised aday card ticket, and provided thread and heap dump. But didn't got any satisfactory response. That's why i have posted over here, is some one else is also facing same issue in 6.1 or not.

Avatar

Avatar
Boost 1
Level 1
cindyp42912329
Level 1

Like

1 like

Total Posts

5 posts

Correct Reply

0 solutions
Top badges earned
Boost 1
View profile

Avatar
Boost 1
Level 1
cindyp42912329
Level 1

Like

1 like

Total Posts

5 posts

Correct Reply

0 solutions
Top badges earned
Boost 1
View profile
cindyp42912329
Level 1

13-01-2017

The same thing happened to our AEM publish instance too. We are running 6.2 with hot fixes.

Did you ever find out what triggered it and what is the solution?

Avatar

Avatar
Validate 10
Level 3
anilkum
Level 3

Likes

3 likes

Total Posts

65 posts

Correct Reply

1 solution
Top badges earned
Validate 10
Validate 1
Boost 3
Boost 1
Affirm 1
View profile

Avatar
Validate 10
Level 3
anilkum
Level 3

Likes

3 likes

Total Posts

65 posts

Correct Reply

1 solution
Top badges earned
Validate 10
Validate 1
Boost 3
Boost 1
Affirm 1
View profile
anilkum
Level 3

14-01-2017

Hi,

Please check if there is an online compaction kicked off by default. If compaction is running then the shutdown will not happen. Also in case there is an reindexing triggered then shutdown will pause till it is completed. These situations I have seen but are not exhaustive,

Hope this helps.

Thanks,

Anil

Avatar

Avatar
Validate 1
Level 3
bobkranson
Level 3

Likes

8 likes

Total Posts

68 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
Applaud 5
View profile

Avatar
Validate 1
Level 3
bobkranson
Level 3

Likes

8 likes

Total Posts

68 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
Applaud 5
View profile
bobkranson
Level 3

17-01-2017

Having seen your post the other day and looking through other things, I wonder if you have looked at how many threads are running?  I found this page in the default environment and noted the 'shutdown gracefully' setting which would be a reason why your system may take too long to shutdown (?!)  In the event there is a runaway thread, you would be waiting for default or internal timeouts to allow the 'graceful' shutdown.

http://localhost:4502/system/console/status-slingthreadpools

Most notable (and I would like some examples to set the 'Adobe-way') are these settings:

~- queue size : -1
- keep alive time : 60000
- block policy : RUN
- shutdown graceful : false
- shutdown wait time : -1

1) How to bound the queue size?  The thread pool might be 5 but you could have a million queued up!

2) Anyway to override the thread pools' shutdown graceful true/false during a stop request?  Me and grace had an argument, I want to kill her or end the relationship slamming the door shut no matter the cost....smile its referring to Process IDs (pids)

3) Anyway to inject a shutdown wait time of some value (2 or 5 mins) after a thread has started or for future threads?  I sent a letter to the IRS on Apr 15 but they haven't got it yet (shutdown notice), my returns will be late but sooner than they send me the late notice and bill....(wait time)

end of day, needed to inject smiles with these questions ;->

Avatar

Avatar
Validate 1
Level 3
bobkranson
Level 3

Likes

8 likes

Total Posts

68 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
Applaud 5
View profile

Avatar
Validate 1
Level 3
bobkranson
Level 3

Likes

8 likes

Total Posts

68 posts

Correct Reply

0 solutions
Top badges earned
Validate 1
Boost 5
Boost 3
Boost 1
Applaud 5
View profile
bobkranson
Level 3

17-01-2017

Having seen your post the other day and looking through other things, I wonder if you have looked at how many threads are running?  I found this page in the default environment and noted the 'shutdown gracefully' setting which would be a reason why your system may take too long to shutdown (?!)  In the event there is a runaway thread, you would be waiting for default or internal timeouts to allow the 'graceful' shutdown.

http://localhost:4502/system/console/status-slingthreadpools

Most notable (and I would like some examples to set the 'Adobe-way') are these settings:

~- queue size : -1
- keep alive time : 60000
- block policy : RUN
- shutdown graceful : false
- shutdown wait time : -1

1) How to bound the queue size?  The thread pool might be 5 but you could have a million queued up!

2) Anyway to override the thread pools' shutdown graceful true/false during a stop request?  Me and grace had an argument, I want to kill her or end the relationship slamming the door shut no matter the cost....smile its referring to Process IDs (pids)

3) Anyway to inject a shutdown wait time of some value (2 or 5 mins) after a thread has started or for future threads?  I sent a letter to the IRS on Apr 15 but they haven't got it yet (shutdown notice), my returns will be late but sooner than they send me the late notice and bill....(wait time)

end of day, needed to inject smiles with these questions ;->