I'm developing an adaptive form for a customer which is submitted to a LC approval workflow. For this I installed Adobe AEM Forms 6.0 on JEE. On this JBOSS there is an author instance of AEM installed automatically. My question now is how do I setup/install the Publish instance? Am I going to need to install a completely new JEE installation or can I deploy the Publish instance on the excisting JEE environment?
You need to install the Publish instance as a plain AEM instance with AEM Forms installed running in the Publish run mode. Your JEE instance should really be your Processing instance and your Author server should point to the JEE instance. The adaptive form should automatically see any processes that have been configured correctly on the Processing server and you can set one of those processes as your submit handler (https://helpx.adobe.com/aem-forms/6/submit-form-data-livecycle-process.html)
You are correct - when you install AEM Forms 6.0 JEE you get AEM 6.0 installed within the JEE stack (which is how Livecycle ES3/4 did it too). That is your processing server in the topology diagram. They have separated them logically but in reality they are the same installation (it's basically a re-branded Livecycle ES4 Installation).
You have to create a separate set of servers for each environment. Don't forget you will also need a Dispatcher in front of the Publish server as well as an optional Dispatcher in front of the Author server if you find you have a lot of content Authors in Production. Usually this isn't required but a nice to have.
If I look at the architecture samples from your link, I think the 'Customers using HTML workspace', would be the one that applies for our case.
But here the installation seems different and they speek of co-deploying AEM and AEM Forms on the same JEE. Which also happens automatically for me when in stalling AEM Forms on JEE.
Now, I have a new question: Does the 'Customer using HTML Workspace' architecture contain the DEV, Test-Acceptance and Production environments? Or does this architecture have to be setup for each environment?