Level 4

Solved

AEM 5.6.1 is crashing some time

Forum|Forum|10 years ago
October 16, 2015
7 replies
2401 views

Spring OSGI services make Web services call to web sphere based external application and lot of 500 and 503 error happens in OSGI Java call with whatever reason from back end web services.. Spring classes from OSGI to Web services is based on https protocal (Secure)..

Manually testing on CQ5 server works very well CQ5 pull information from Web services all works well.

Automatic load testing on CQ like 200 concurrent users for an hours also looks good but when same testing carry our few times in week ...Very Randomly any one of CQ5 publish get crashed and doesn't show any evidence like PermGen, Out of memeory or Diskspace. Log also doesn't have any evident why it crashes ... I see only three kind of error ...500,503 and this SSL exception

0.09.2013 15:31:23.582 *WARN* [172.22.20.105 [1378845083565] <parse>] servletengine I/O error in execution: {} javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 88
   at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476)
   at sun.security.ssl.AppInputStream.read(AppInputStream.java:92)
   at java.io.InputStream.read(InputStream.java:101)
   at com.day.j2ee.servletengine.ServletHandlerImpl.parseRequest(ServletHandlerImpl.java:450)
   at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:301)
   at com.day.j2ee.servletengine.HttpListener$Worker.run(HttpListener.java:644)
   at java.lang.Thread.run(Thread.java:722)
Caused by: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 88
   at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
   at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
   at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:974)
   at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
   at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
   at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:2173)
   at com.day.j2ee.servletengine.ServletHandlerImpl.getCipherSuite(ServletHandlerImpl.java:978)
   at com.day.j2ee.servletengine.RequestImpl.recycle(RequestImpl.java:269)
   at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:289)
   ... 2 more
Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 88
   at sun.security.ssl.CipherBox.removePadding(CipherBox.java:684)
   at sun.security.ssl.CipherBox.decrypt(CipherBox.java:423)
   at sun.security.ssl.InputRecord.decrypt(InputRecord.java:154)
   at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:969)
   ... 8 more
10.09.2013 15:31:23.692 *WARN* [172.22.20.105 [1378845083692] GET /servic except 500, 503 and this SSL exception.

My doubt is on this SSL exception is crashing CQ5 JVM.. Please share your experience.

Thanks Chandra

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by Peter_Puzanovs

Try AppDynamics, it allows you to monitor the transaction path in details, when fails you can dig deep and figure out the exact method that fails and exact path it fails on.

smacdonald2008

Level 10

Can you please provide more information on what you are trying to perform here.

A

AshokPelluru

Getting same error

<parse>] servletengine I/O error in execution: {} javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 194

at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)

at sun.security.ssl.AppInputStream.read(Unknown Source)

at java.io.InputStream.read(Unknown Source)

at com.day.j2ee.servletengine.ServletHandlerImpl.parseRequest(ServletHandlerImpl.java:466)

at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:317)

at com.day.j2ee.servletengine.HttpListener$Worker.run(HttpListener.java:625)

at java.lang.Thread.run(Unknown Source)

Caused by: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 194

at sun.security.ssl.Alerts.getSSLException(Unknown Source)

at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)

at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.getSession(Unknown Source)

at com.day.j2ee.servletengine.ServletHandlerImpl.getCipherSuite(ServletHandlerImpl.java:947)

at com.day.j2ee.servletengine.RequestImpl.recycle(RequestImpl.java:272)

at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:309)

... 2 more

Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 194

at sun.security.ssl.CipherBox.removePadding(Unknown Source)

at sun.security.ssl.CipherBox.decrypt(Unknown Source)

at sun.security.ssl.InputRecord.decrypt(Unknown Source)

any help?

A

AshokPelluru

Getting same Error

<parse>] servletengine I/O error in execution: {} javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 194

at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)

at sun.security.ssl.AppInputStream.read(Unknown Source)

at java.io.InputStream.read(Unknown Source)

at com.day.j2ee.servletengine.ServletHandlerImpl.parseRequest(ServletHandlerImpl.java:466)

at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:317)

at com.day.j2ee.servletengine.HttpListener$Worker.run(HttpListener.java:625)

at java.lang.Thread.run(Unknown Source)

Caused by: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 194

at sun.security.ssl.Alerts.getSSLException(Unknown Source)

at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)

at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.getSession(Unknown Source)

at com.day.j2ee.servletengine.ServletHandlerImpl.getCipherSuite(ServletHandlerImpl.java:947)

at com.day.j2ee.servletengine.RequestImpl.recycle(RequestImpl.java:272)

at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:309)

... 2 more

Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 194

at sun.security.ssl.CipherBox.removePadding(Unknown Source)

at sun.security.ssl.CipherBox.decrypt(Unknown Source)

at sun.security.ssl.InputRecord.decrypt(Unknown Source)

Chandra_guptaAuthor

Level 4

Its Spring OSGI bundle that has certain spring based custom java services makes https call to extrenal price related web services ..consumes JSON outpout.

Peter_Puzanovs

Accepted solution

Community Advisor

Try AppDynamics, it allows you to monitor the transaction path in details, when fails you can dig deep and figure out the exact method that fails and exact path it fails on.

Chandra_guptaAuthor

Level 4

I have almost identified issue.

Oracle JDk 7.21 for linux has bug that carch JVM when java class establishes https connection to any external services. this does not happen over http.

Oracle JDK 7.25 has fixed this issue... we are in process of applying that patch and update this forum.

Thanks Chandra

Chandra_guptaAuthor

Level 4

JDK update has not helped..

When I get CQ running on http and external web services since CQ is sending non-encrypted request packet to extreanl web services and that has no issue but as soon i get working CQ on HTTPS... after every few hundred call to web services CQ logs shows this exception

------------------------------------------------------------------------------------------------

8.09.2013 16:12:17.985 *INFO* [10.100.183.102 [1379538737984] G 18.09.2013 16:12:18.481 *WARN* [10.100.183.102 [1379538738465] <parse>] servletengine I/O error in execution: {} javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 156 at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476) at sun.security.ssl.AppInputStream.read(AppInputStream.java:92) at java.io.InputStream.read(InputStream.java:101) at com.day.j2ee.servletengine.ServletHandlerImpl.parseRequest(ServletHandlerImpl.java:450) at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:301) at com.day.j2ee.servletengine.HttpListener$Worker.run(HttpListener.java:644) at java.lang.Thread.run(Thread.java:724) Caused by: javax.net.ssl.SSLHandshakeException: Invalid Padding length: 156 at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:974) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339) at sun.security.ssl.SSLSocketImpl.getSession(SSLSocketImpl.java:2173) at com.day.j2ee.servletengine.ServletHandlerImpl.getCipherSuite(ServletHandlerImpl.java:978) at com.day.j2ee.servletengine.RequestImpl.recycle(RequestImpl.java:269) at com.day.j2ee.servletengine.ServletHandlerImpl.process(ServletHandlerImpl.java:289) ... 2 more Caused by: javax.crypto.BadPaddingException: Invalid Padding length: 156 at sun.security.ssl.CipherBox.removePadding(CipherBox.java:684) at sun.security.ssl.CipherBox.decrypt(CipherBox.java:423) at sun.security.ssl.InputRecord.decrypt(InputRecord.java:154) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:969) ... 8 more 18.09.2013 16:12:18.577 *WARN* [10.100.183.102 [1379538738576] GET /services

------------------------------------------------------------------------------------------------------

Defect has been reported oracle bug

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8013059

I am thinking to apply RSA cipher in CQ... but not sure how to do that?

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded