we are not able to login to our instance for 5 days. An error is thrown: SOP-330012 Empty return for method 'ExecuteQuery' of service 'xtk:queryDef or console is in status Waiting connection until session expire. We are referenced on Tomcat because our Unix team confirmed that server does not listen on localhost:8080 even if it is set on Tomcat configuration. It is weird that there is no any changes form our side or server team. Just stopped working and no interesting logs. No java updates.
Please check output difference for command nlserver pdump -who for working environment (connection bottom are presented):
and nonworking one (cannot present connections):
Nothing listen on port 8080 regarding other applications. Process are stuck as it takes time to stop/start them. Nothing interesting in logs.
Telnet localhost 8080:
We tried to change port and did not helped. We performed several restart of processes and got application to work for 6 hours once and 5 minutes second time.
We tried with backup folder of etc/local/neolane/nl6 and situation persist. Tried with firewall shutdown and without success.
I would appreciate any crazy idea because we tried almost everything.
The ticket is already open and AC Support agreed with us that localhost does not listen on 8080. We used strace tool to generate logs when starting web@default and we are waiting for feedback. We compared internaly those logs with logs from other servers where localhost listen on 8080 and we did not find bigger difference. We used the backup from whole nl6 folder and that did not helped, therefore my thougths it is related to something which is outside from nl6 folder like server, java, httpd... No java upgrades, server is fine (firewall turned off) and still nothing helps. I will check LDAPS. Point here is that localhost8080 cannot be started at all and that port is not used by other apps, but cannot find why. We tried with older backup of serverConf.xml and nothing. Yes, we use consoles from our PCs. Not able to connect from browser also just blank page. Apache available as /r/test page.
Please firstly create a P1 priority (critical) ticket in Adobe Campaign extranet support.
Then, in parallel you can investigate by your own and ask for forum community help, but it seems to be tricky, if you are very sure nothing has been changed in configuration, either network, or system, or serverConf.xml or config-yourinstance.xml files. (For the latter check for serverConf.diff.xml file). Do you have Active Directory (LDAP) for authentication ? or IMS ?
One not so "crazy" idea: are you not able to connect from client (Windows client) directly installed on the server or from your PC ?
Because, if you set the connection url to use :8080 port you bypass IIS and goes directly to Tomcat. So if you use the :8080 url on the server itself you won't be blocked by firewall.
But perhaps the daemon or port is not open anymore ? Please ask to your network team to do a netstat command to see if 8080 is listed.