how to find out what version of java is running on cloud AEM?

[TB3dock]

Since launching our sites a year ago, we have suffered constant issues with calling APIs and 3rd parties from servlets.  This includes GOAWAY being randomly received, to invalid headers being appended and different Envs having different java http request characteristics with the same code.

Is there any way to find out what version of Java is running on a particular env?  There doesn't seem to be anything in cloud manager or dev console. 

We have seen for example that openjdk 11.0.14 has some issues with http2 for example.

[Anmol_Bhardwaj]

AEM as a Cloud Service only supports Java 11+.

And if you try to deploy code with, say Java 8, it won't work. It will throw error an bundles will not get resolved.

[TB3dock]

Hi, thanks for the reply, but we need to know exactly which version 11 is being used. e.g. oracle or openjdk.  11.0.10 or 11.0.14 etc.

This is because our site went down for some time, and the only explanation is that Adobe are using 11.0.14 openJDK, which has a known bug. However, Adobe should be using Oracle JAva, which is more reliable.

[Jörg_Hoh]

Do you have a link for that bug you are referring to?

[TB3dock]

https://access.redhat.com/documentation/en-us/openjdk/11/pdf/release_notes_for_openjdk_11.0.14/openj...

What we are seeing from AEM cloud is its incorrectly setting both host and authority headers, which is a major bug only present in open JDK 11.0.14.  Because it sets both headers, when our servlets try to call an api with nginx in front of it, nginx responds with 400 due to malformed request.  This has caused us significant downtime.  Adobe should be using java JDK, not the less reliable openjdk.  One major downside of cloud AEM is that we cant see anything, not even what jdk is being used.  

CHAPTER 3. OPENJDK 11.0.14.1 RELEASE NOTES Review the following release notes to understand changes from this OpenJDK 11.0.14 patch release: 3.1. RESOLVED HTTPS://GOOGLE.COM CONNECTION ISSUE The OpenJDK 11.0.14.1 release resolves an issue that was identified when using the Java HTTP client, java.net.HTTPClient, to connect to the https://google.com URL. This issue persisted on the OpenJDK build for Microsoft Windows and on the OpenJDK build for RHEL. The initial OpenJDK 11.0.14 release contained a regression that was introduced by improvements to the HTTP client. This regression caused both the :authority and the Host header fields to be sent in HTTP/2 requests, which are rejected by some HTTP servers, such as Google’s server. When you attempted to establish this connection, you would receive an exception message. This exception message would indicate that the Java HTTP client could not successfully communicate by using the HTTP/2 protocol. Example of an exception message when attempting to connect to https://google.com with java.net.HTTPClient The OpenJDK 11.0.14.1 release resolves the issue by reverting to the original behavior of only setting the :authority header field to be sent in an HTTP/2 request.

[TB3dock]

also: https://access.redhat.com/errata/RHBA-2022:0733

[TB3dock]

Adobe admitted that they updated production to oracle java 11.0.14 which has a well documented and already fixed P1 bug.   This took down our sites for around 10 hours as we did not know where the problem came from.  We hotfixed our prod env by downgrading http requests to http 1.1.  We could not reproduce in our dev cloud envs as adobe updated prod before they updated dev. They updated dev 2 days later to the same broken version of java.

In our industry, 10 hours of down time can cost anything from  £100k to £1m

AEM cloud is now looking like not an option for enterprise commerce systems.

We have now created a servlet which reports the java version, and we will setup a external system to check this every 5 minutes.  When Adobe change the production java version without telling us, at least we will get an alert so can get a standby team woken up to do an emergency smoke test, and get developers out of bed if it breaks the sites like this one did.

If only Adobe would send out a warning with detailed change info to its partners and customers when it's going to make major untested infrastructure changes, or at the very least, do non prod before it does prod.

[kautuk_sahni]

@TB3dock 

[Webinar | AEM GEMs] Looking under the hood - Cloud Manager 2022 | Register: https://adobe.ly/3t4jfgp  & Ask Questions related to this Webinar: https://adobe.ly/3O0rdzd 

Date &Time: Wednesday, June 15, 2022 - 8 am PDT OR 5 pm CET OR 8.30 pm IST
Speakers: Remus Stratulat & Shankari Panchapakesan

Share this within your Organisation and with your AEM peers!!