Expand my Community achievements bar.

SOLVED

AEM 6.1 Startup Error: Missing Authentication Service due to missing blob

Avatar

Level 4

Hi,

We recently upgraded our CQ 5.6.1 to AEM 6.1 with Mongo DB.  Our upgraded AEM 6.1 is throwing the "Missing Authentication Service" error.  The error.log file is saying that a blob could not be found and a call was issued to shut down the repository.  How do we delete the blob Id in Mongo DB (assuming that's where it is getting the Id and complaining that the blob could not be found in the file system at crx-quickstart/repository/repository/datastore)?  

Thanks in advance.

1 Accepted Solution

Avatar

Correct answer by
Employee Advisor

From the logs it seems the error is thrown when a particular blob file is not found. It could be that the file was deleted erroneously or it does not exist. You can try the following steps for recovery but I would recommend you to contact Adobe daycare support team for analyzing the root cause and providing you the right fix. 

  1. Go to crx-quickstart/repository/repository/datastore folder on the file system. 
  2. Under the datastore folder, if the following path does not exist - /2f/c5/21 then create this path manually.
  3. Inside the above path, add a dummy binary file(empty file) and just name it as "2fc521266a17b55fecf8dc95d4ffb455ada35e05". (It would be to get the original file from any past backups though)
  4. Restart AEM and see if the exception disappears.  

Also, I found that this exception is thrown when there are errors in reading the datastore files. There is already a JIRA logged for this - https://issues.apache.org/jira/browse/OAK-3442

View solution in original post

7 Replies

Avatar

Level 10

I sent this to our support team to see if this is a known issue. 

Avatar

Employee Advisor

Did you start your AEM instance with the following run modes - 

  • -r crx3,crx3mongo for MongoMK

Can you share the contents of following config files in crx-quickstart/install folder? Also ensure that there are no spaces in the paths of your aem installation folder. 

-org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.cfg

- org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.cfg

 

There is a known issue with Windows environment -

  • Update to 6.1 from 5.x and 6.0+CRX2 fails on Windows due to backslashes written to FileDataStore.cfg. Workaround is to modify the 'org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.cfg' file with datastore path before we run Migration command (CQ-42291)

Avatar

Level 4

Hi,

The following shows the "java process" after we run our start script

java -server -Xmx4G -Xms4G -XX:MaxPermSize=2G -XX:PermSize=2G -Djava.awt.headless=true -Dsling.run.modes=author,crx3,crx3mongo -jar crx-quickstart/app/cq-quickstart-6.1.0-standalone-quickstart.jar start -c crx-quickstart -i launchpad -p 4502 -Dsling.properties=conf/sling.properties

The run modes are correct

The config files' contents are as follow

-----------org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore.cfg----------

 

path=./crx-quickstart/repository/repository/datastore
minRecordLength=4096

---------org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService.cfg-----------

customBlobStore=true
mongouri=mongodb://localhost:47017/
db=aem6

 

We do not have a SegmentNodeStoreService.cfg file...The above cfg files were created by the upgrade process

Thank you

Avatar

Employee Advisor

Can you check whether the AEM process has read/write privileges on datastore folder and can you share the exception stack trace/log file ?  

Avatar

Level 4

Hi,

The AEM was started with root and the datastore folder has rwx rights for root.  I updated the rights to the datastore folder and its contents to 777, just to test if it's permission issues.  After restarting AEM, I still see the same error.

The full warnings and exceptions logged in error.log are in the attached text file

Avatar

Correct answer by
Employee Advisor

From the logs it seems the error is thrown when a particular blob file is not found. It could be that the file was deleted erroneously or it does not exist. You can try the following steps for recovery but I would recommend you to contact Adobe daycare support team for analyzing the root cause and providing you the right fix. 

  1. Go to crx-quickstart/repository/repository/datastore folder on the file system. 
  2. Under the datastore folder, if the following path does not exist - /2f/c5/21 then create this path manually.
  3. Inside the above path, add a dummy binary file(empty file) and just name it as "2fc521266a17b55fecf8dc95d4ffb455ada35e05". (It would be to get the original file from any past backups though)
  4. Restart AEM and see if the exception disappears.  

Also, I found that this exception is thrown when there are errors in reading the datastore files. There is already a JIRA logged for this - https://issues.apache.org/jira/browse/OAK-3442

Avatar

Level 4

Hi,

I created the dummy binary file and the log states that the CRX Repository is started.  There were some additional errors related to oak:lucene and we are still unable to access the instance.  I'll wait a bit and see if AEM ever comes up.  

I have a ticket open with Support.  If they ever provide any "official" way to resolve blob related errors, I'll update here.

Thanks for your fabulous help!