Workflow services are taking 50 min to start.

[Former Community Member]

Hello,



One of my colleagues was on site and he installed the turnkey install of Form Manager and WorkFlow on a newly built server.



The client is doing some tests which required them to restart the server. They have noticed that the WF services take an excessively long time to start. Sometimes it takes up to 50 minutes.



I don't have very much info or other details at this point, but I was wondering if anyone had any insight as to why this may be happening. What should I look at when I get the opportunity to look at their server? Are there any known issues regarding this? Has anyone else had this type of issue? If so, how did you overcome it?



Thanks,

Stone_Golem

[Former Community Member]

Not many ideas without any info. Try taking a look at the server.log from startup till it completes and see what's going on.



One thought, if the client is putting a lot of custom fonts into the Font Manager module it can get extremely large and make start up quite long.



Chris

Adobe Enterprise Developer Support

[HowardTreisman]

Another thought - a lot of what startup does is unpack jar files - if you've got a problem on your hard disk (eg lack of space, badly fragmented), it can cause things to run really slow. However, 50 minutes does seem extremely excessive.



Chris - what are the fonts in Font-manager actually used for? And what are the implications of removing it completely?



Thanks,

Howard

[Former Community Member]

The fonts in font manager are used on the server side by LiveCycle forms when it renders an XDP into a PDF. Since fonts are not embedded into XDP files you need to add any custom fonts you use into the font manager so the PDF is rendered properly.



I don't know what would happen if you removed it totally. I imagine Forms will try to use it and then throw a bunch of errors. If you only use forms to render PDF's, not XDP's, you may have no trouble at all if you don't deploy it. Keep in mind this is all theoretical off the top of my head, I've never tried, and certainly wouldn't recommend, running without it.



Chris

Adobe Enterprise Developer Support

[HowardTreisman]

Thanks Chris



The reason I ask is another poster was getting problems with font-manager loading, and I suggested he remove it, just to get through the JBoss startup. Obviously he'll have to resolve his problems in the longer term.



Howard

[Former Community Member]

Thanks guys. I'll give it those suggestions a whirl.



Is there a general rule of thumb for how long it should take for the services to start? I usually give it 10 minutes or so and then I'm in business. Does anyone go by something different?



Thanks in Advance,

[HowardTreisman]

For Form Manager and Workflow, I've found it usually takes between about 3-5 minutes, depending on the machine.



Howard

[Former Community Member]

Hi All,

I am also experiencing same delay in start of server I had mad four installation on Windows 2003 Server and 3 server are facing the same problem I thought it might be the problem with my server but now I think it is something else

[Former Community Member]

The amount it time it takes to start totally depends on the machine and how many applications the app server needs to deploy. For a decent machine, with no custom fonts in the font manager, 10 minutes should be a safe estimate.



When you're testing you may want to start JBoss from the command line instead of as a service. That way you can see messages in the console as they happen instead of having to read through the log afterwards.



Chris

Adobe Enterprise Developer Support

[HowardTreisman]

For watching the log file, here's a great tool that makes this easier.



http://www.baremetalsoft.com/baretail/index.php



Howard

[Former Community Member]

Check your version of the Java SDK. You must specifically be running nothing newer than Java 1.4.2_08. I had the same problem. Everything "worked" with version 1.4.2_10, but took over an hour to boot up. I couldn't even install all of the LiveCycle products because the Configuration Manager would give up on the JBoss service before it finished booting.



Once I downgraded to version 1.4.2_08, it boots up in under 5 minutes with Reader Extensions, Workflow Server, Form Manager, and Barcode Decoder services installed. You can even install 1.4.2_08 in the same directory as your existing 1.4.2_10 and reboot without having to reinstall any part of the LiveCycle package.



Clifton Cline

[Former Community Member]

Thanks Clifton.



I am the "colleague" that Stone refered too. Since then, we did a re-install with manual deployment and it solved the start up issue. (And we did use 1.4.2_10.)



Now we have a new and unusual problem. It appears that JBoss does not always deploy the LiveCycle.ear file each time the service starts. After a few restarts of the service the environment is only useable 50% of the time. Has anybody else witnessed any 404 errors after rebooting the JBoss service? After checking the log, I can tell it is not deploying the LiveCycle.ear file. I have also reproduced this problem with a 1.4.2_8 version of the SDK.



Thanks,



NetKeeper

[HowardTreisman]

Can you explain what you mean by "not deploying"?

Are you getting an exception in the log file?

Howard

[Former Community Member]

Howard,



Yes! We are getting some exceptions in the log.



If I watch the loading of JBoss I can tell when it will fail. (I watch the tmp folder as it is deploying components) On a "good" boot it takes several minutes to deploy the massive LiveCycle.ear file. When a failure occurs it takes only a minute or so. From what I can tell from the log file, it appears to be failing on the forms_admin_help_ja.war and then again on the jmx_admin_help_fr.war components.



It usually takes me around 5 consectutive starts of JBoss to see the error, but at the clients it happens 2 out of three times.



Thanks,



Peter L



Here is a portion of a log that indicates the failure:



2006-03-02 16:56:31,953 DEBUG [org.jboss.deployment.MainDeployer] End deployment start on package: deploy.last

2006-03-02 16:56:31,953 DEBUG [org.jboss.deployment.MainDeployer] Deployed package: file:/C:/Adobe/LiveCycle/jboss/server/all/deploy/deploy.last/

2006-03-02 16:56:31,953 DEBUG [org.jboss.deployment.scanner.URLDeploymentScanner] Watch URL for: file:/C:/Adobe/LiveCycle/jboss/server/all/deploy/deploy.last/ -> file:/C:/Adobe/LiveCycle/jboss/server/all/deploy/deploy.last/

2006-03-02 16:56:31,953 ERROR [org.jboss.deployment.scanner.URLDeploymentScanner] Incomplete Deployment listing:

Incompletely deployed packages:

org.jboss.deployment.DeploymentInfo@9accaad9 { url=file:/C:/Adobe/LiveCycle/jboss/server/all/deploy/LiveCycle.ear }

deployer: org.jboss.deployment.EARDeployer@8ddc4c

status: Deployment FAILED reason: URL file:/C:/Adobe/LiveCycle/jboss/server/all/tmp/deploy/tmp14242LiveCycle.ear-contents/services.war/ deployment failed

state: FAILED

watch: file:/C:/Adobe/LiveCycle/jboss/server/all/deploy/LiveCycle.ear

lastDeployed: 1141336577250

lastModified: 1141336546296

mbeans:



MBeans waiting for other MBeans:

ObjectName: user:service=QuartzService,name=QuartzService

state: CONFIGURED

I Depend On: jboss.jca:service=LocalTxCM,name=QuartzDS



Depends On Me:

ObjectName: jboss.web.deployment:war=services.war,id=1012563003

state: FAILED

I Depend On:

Depends On Me: MBeanException: org.jboss.deployment.DeploymentException: URL file:/C:/Adobe/LiveCycle/jboss/server/all/tmp/deploy/tmp14242LiveCycle.ear-contents/services.war/ deployment failed

Cause: org.jboss.deployment.DeploymentException: URL file:/C:/Adobe/LiveCycle/jboss/server/all/tmp/deploy/tmp14242LiveCycle.ear-contents/services.war/ deployment failed

ObjectName: jboss.web.deployment:war=forms_admin_help_ja.war,id=-2112049534

state: CREATED

I Depend On:

Depends On Me:

ObjectName: jboss.web.deployment:war=fm_help_ja.war,id=1073731034

[HowardTreisman]

Hi

I've never seen that before, sorry.

The portion of the log file you've sent is the summary you get right at the end - the "real" error may very well be higher up in the log file. Please take a look and see if there's anything obvious, and post it.



Also, services.war file mentioned above is the web-services user interface. Is it possible you have installed any other war/jar/ear files within the application server (such as Apache axis or others) that may be conflicting with this one? Just a thought?



Howard

[HowardTreisman]

Also, are you using the default JBoss turnkey install, or are you installing your own version of JBoss.



Howard

[Former Community Member]

Our own install of JBoss 3.2.5.



Peter L

[HowardTreisman]

Personally, I'd give it a go on turnkey version of JBoss, and see if that helps. They do several tweaks to the standard install.



Also, please try to find the root cause of the above log file message earlier up in your log file.



Howard

[Former Community Member]

We had done a manual deployment only because we were attempting to use a MS SQL Server database. I have solved the problem we were having with instability of JBoss by doing a turnkey installation of Form Manager, Workflow and Forms. Once I was sure that everything was good, I changed the database connection components.



MS Jar files

Run.bat

Adobe-ds.xml



Then I re-ran the ConfigurationManager, deployed the new LiveCycle.ear file and then initialized the MS SQL database. Finally everything appears to be running stable. This leads me to believe that (like you said) the turnkey does some extra tweaking of JBoss that may not be discussed in the manual deployment instructions.



Thanks,



Peter L

[HowardTreisman]

Great that you're up and running.

I always use the procedure you've described to get things started, and then manually switch over to another database. I'm not sure whether it does some steps that aren't mentioned in the docs, but it sure is easier.



Howard