AEM 6.5 - 503 error every time server is initialize

[mahinder]

I am facing 503 AuthenticationSupport service missing. Cannot authenticate request error every time server is started. This happened many times. There is no such broken error information on the error log file. I have even deleted the repository but didn't help much. I even use oak but still no use. I had set up it multiple times to skip this error but I don't feel confident with this approach. For better information, I am attaching the latest error log file. 

https://drive.google.com/file/d/1aXEcZP_Tvk1ebBxfOmR9Q1ffk2KLeW66/view?usp=sharing 

NOTE: I am a beginner in AEM. Please be gentle 

[ManviSharma]

Hi

Some quick pointers:-

1. Make sure all bundles are installed and running? Start any bundles that do not have the “ACTIVE” status and this should resolve the issue. Verify the same from http://dev.day.com/content/dam/day/onlinetool/COI.html

2. Check all your XML config files for CQ, especially if you remember editing one, "Content is not allowed in prolog" is a common error when the XML is malformed, or you have whitespace before the opening element of an XML file.

Link:- http://help-forums.adobe.com/content/adobeforums/en/experience-manager-forum/adobe-experience-manage...

Link:- http://blogs.adobe.com/dmcmahon/2012/06/18/cq-5-5-update-1-service-unavailable-authenticationsupport...

Link:- https://edivad.wordpress.com/2013/04/02/aemcq5-authenticationsupport-service-missing/

I hope this would help you.

[mahinder]

Hi @ManviSharma ,

Thanks for your suggestions. I have already implemented 1st many times and its shows all active status. For 2nd point, I didn't edit any XML files. I am following https://experienceleague.adobe.com/docs/experience-manager-learn/getting-started-wknd-tutorial-devel... tutorial and didn't change anything. 

Link:- https://edivad.wordpress.com/2013/04/02/aemcq5-authenticationsupport-service-missing/ I have already applied the solution mentioned in this post. Unfortunately, nothing works for me.

[Raja-kp]

@mahinder 

1. Check whether your Apache Sling Authentication Service Bundle is active or not. (http://localhost:4502/system/console/bundles/)- if not active then try to activate it manually.
2. Stop the AEM instance - Go to folder crx-quickstart/repository/remove folder index -> Start the AEM
3. Go for a re-indexing - http://www.wemblog.com/2011/12/how-to-rebuild-index-in-cq5-wem.html
4. Check the ERROR log - if you are seeing any memory related logs (Java Heap Space (java.lang.OutOfMemoryError: Java heap space)) - if yes, then try to increase the memory value in the JVM start script.

Regards,

Raja

[mahinder]

Hi @Raja-kp ,

1. Check whether your Apache Sling Authentication Service Bundle is active or not. (http://localhost:4502/system/console/bundles/)- if not active then try to activate it manually.

=> Already active

2. Stop the AEM instance - Go to folder crx-quickstart/repository/remove folder index -> Start the AEM

=> Done but didn't help. 

3. Go for a re-indexing - http://www.wemblog.com/2011/12/how-to-rebuild-index-in-cq5-wem.html

=> I used oak for finding the error in the repo. In the above line, I am not able to install the tool for reindexing as it's still showing errors.

4. Check the ERROR log - if you are seeing any memory related logs (Java Heap Space (java.lang.OutOfMemoryError: Java heap space)) - if yes, then try to increase the memory value in the JVM start script.

=> I have already attached the error log file. Would you please check it once? I did too and didn't find such an error.

NOTE: AEM version: 6.5

[Siva_Sogalapalli]

@mahinder 

What is the java version you're using? 

If setting up new instance is an option for you, then you can try below and see how it works. 

1. Delete existing crx-quickstart folder

2. First time install AEM using command line :   java -jar <aem-author-4502>. jar

3. One the AEM instance is up, later on you can use any options to start AEM.

[mahinder]

@Siva_Sogalapalli 
I am using Java 11.0.4 version. I understand the reset-up concept but I am quite curious about what will happen to existing projects. I am new to AEM, so I am not able to judge whether it's a good approach or not. What if I will be working on some live projects and then such an error will encounter. If I will delete the crx-quickstart folder and reconfigure everything from scratch, then my progress will be lost. Please correct me if I am wrong.

[Siva_Sogalapalli]

Yes, if you delete crx-quickstart folder for any reason, then we would need to configure everything form scratch. NOTE: Please remember that it is not advisable/solution to fix on prod & any other instance especially removing crx folder and install again. 

In general, there could be multiple reasons for these kind of issues and below are the few debug steps:

1. Make sure you're compatible java version. 

https://experienceleague.adobe.com/docs/experience-manager-learn/foundation/development/set-up-a-loc... 

2. Make sure all the bundles are up & running 

3. Check the logs if you can find any errors and take actions based on errors. 

[mahinder]

@Siva_Sogalapalli Right. I did all the steps you have mentioned but didn't resolve the issue. That's why I posted here to get some help on debugging. Deleting crx-quickstart folder is not the correct solution.

[ManviSharma]

Could you please check the disk usage as well in the server as well as through the /etc/reports/diskusage.html.

This way we can know we have the space but it is something problem with the segments. Also, if the segments get corrupted and we have sufficient disk space We need to run the segment consistency check once to confirm if there are any good revisions available to revert to refer[0].

[0] https://experienceleague.adobe.com/docs/experience-cloud-kcs/kbarticles/KA-16542.html?lang=de-DE

[mahinder]

@ManviSharma When I am trying to check the disk usage i.e /etc/reports/diskusage.html, it throws an error:

diskusage

I already used oak to check the good revision of segment store following the https://experienceleague.adobe.com/docs/experience-cloud-kcs/kbarticles/KA-16542.html?lang=de-DE. However, there are no corrupt entries beyond the latest good revision return by oak. This means the segment store is not corrupted.

That's why I am not able to debug the real issue. I already followed the two approaches i.e deleting index file from the repo and oak detection of the last good revision entry. Nothing seems to be working.

[mahinder]

@ManviSharma @Siva_Sogalapalli @Raja-kp Any more suggestions? Please share. The issue is still not resolved.

[Siva_Sogalapalli]

@mahinder I see multiple errors in the log which you've shared and have few questions.

1. Is it your fresh AEM instance and you run into this while setting up?

2. Or you got this issue upon any activity like upgrading any packages/bundles or upon restart your machine without proper stop etc.

3. I see issues related to blob storage, oak index, core bundles etc.. 

Team has already shared few debugging steps, if none of those works for you then i would suggest below.

1. Don't just hang on this issue, create new instance and start working on your day to day work as it's your local. 

2. Keep the corrupted environment in backup and try debugging whenever you get time. Please share below info for further help: 

    A. Please clear/delete existing logs 

    B. Restart AEM & Access localhost:4502 and share all the log files which would help to analyse the issue specific to your local. 

Please note that the moment you get hand-on experience on debugging, you can easily fix these kind of issues on any other environment by yourself or community would be able to help you or you can even raise Adobe support ticket if you've access for quick help, but make sure to keep track of all the activities like upgrades or core package upgrades, logs etc when you face such issues. 

Hope this would help you.