Server side error encountered. Please contact AEM administrator - AEM 6.5 on premise

[aemuser001]

Hi,

An instance of AEM 6.5.4 is running on Windows OS. The crx respository is in Oracle database, and the following message is seen in the error.log

12.01.2023 00:00:00.375 *ERROR* [sling-default-3-AdobeSignRefreshTokenScheduleJob] com.adobe.forms.foundation.oauth.model.OAuthConfigSlingModel Refresh Token not present for: /conf/global/settings/cloudconfigs/fdm/ms-dynamics-odata-cloud-service/jcr:content

...

[sling-default-3-health-org.apache.sling.discovery.oak.SynchronizedClocksHealthCheck] org.apache.sling.discovery.oak.SynchronizedClocksHealthCheck execute: clocks in local cluster out of sync by 1000msms which is equal or higher than the low-water mark of 1000ms.

Earlier in the last few months, the HTTP 503 error was displayed when accessing the admin console from /aem/start.html. On restarting AEM instance the issue will go away. Now receiving the error 'Server side error encountered. Please contact AEM administrator', and restart of AEM instance is not resolving this issue. The issue appears to be with the UI display. I can access crx/explorer/index.jsp.

One suggestion given to us earlier was to try to sync the time between the oracle DB and AEM server clock. My DB team informed that the clocks are in sync.

Any help on how to resolve it will be great.

Thanks,

Leena

[Pulkit_Jain_]

@aemuser001 

As per the sling docs[0], SynchronizedClocksHealthCheck class checks how much the local time differs from the DocumentStore's time. It then applies low- and high-water marks to that time difference:

• if the value is higher than the high-water mark (5sec by default), then it issues a critical
• if the value is lower than the high-water but higher than the low-water mark (1sec by default), then it issues only a warning
• if the value is lower than the low-water mark, then it issues only an info

Based on the error msg, you're seeing a warning. Can you check if the time difference (if any) between the local server and DB time is lesser/greater than 1 sec?

Do you see this message whenever you try to render the AEM home page or after a periodic interval? Any other msgs when you try to render the page?

[0] - https://sling.apache.org/apidocs/sling12/org/apache/sling/discovery/oak/SynchronizedClocksHealthChec... 

[aemuser001]

...

[Pulkit_Jain_ · Accepted solution]

@aemuser001 

Thanks for sharing the ticket#.

I will review the error logs shared.

[aemuser001]

...

[Pulkit_Jain_]

@aemuser001 

I have reviewed the logs shared and could see the posted error msgs[0].

These two tables: DATASTORE_DATA and DATASTORE_META contains information for a blobId reference so you can restore the database state to check if the issue is resolved. 

I recall now, we have had a similar discussion sometime back, why are you seeing such issues on this setup so frequently- Could be a disk space outage, or network share instability?

[0] - 

19.01.2023 07:54:38.485 *ERROR* [10.3.248.58 [1674132878282] GET /libs/granite/core/content/login.html HTTP/1.1] com.adobe.granite.ui.clientlibs.impl.HtmlLibraryManagerImpl Cannot read input stream
javax.jcr.RepositoryException: java.io.IOException: java.lang.RuntimeException: failed to read block from backend, id 1a62282beed54f062f8202305d4e3af4e9d1cb7a7fe5727a4506afb34d06822d@0

[aemuser001]

Hi Pulkit,

Thanks for providing the solution. 

Leena