JDK 11.0.14 has a well known major P1 bug where it incorrectly inserts both host and authority header with http 2.0 requests, causing http 400 malformed exception responses when hitting nginx and other API providers (notably google)
This is the bug in 11.0.14:
It has been patched some time ago in 11.0.14.1
On monday 28th March 2022 at 12:30, adobe changed the java version to the unpatched 11.0.14, on production, without telling us, taking down all our production sites - we were suddenly getting 504 for all the api calls to various backend systems made from within AEM. As we had not changed anything, and there was no restart in the logs, it was extremely hard to debug this, and we were down for a day, losing significant revenue and customer confidence. We created support tickets with Adobe, but it took them 2 days to respond. Two days down would be catastrophic.
We are absolutely shocked that adobe would change the java version on our production cloud instances without:
1) letting us know
2) doing it on non production (dev envs) first. Dev wass still on an earlier version.
3) using a java version with a know bug and a known patch.
4) Adobe appear to be using openJDK, which is less stable than the oracle java, although it's not possible for us to verify or find out what version Adobe are using.
5) We dont know a way to find out what version Adobe is using on our production sites, so we cant create alerting on our end if Adobe change it, and to query it when there is a production issue to see if its a bug in that version of java.
Our workaround was to do an emergency hotfix to production downgrading http2 to http1.1.
This is absolutely unacceptable, and now we have to strongly consider going on prem, although we don't have staff to support or manage it.
