How to connect a Dispatcher to a Load Balancer where the LB is in turn connected to other dispatchers

[Preetpal_Bindra]

Hello Guys, Need your suggestions on a Apache/Dispatcher setup. (Please see the attached diagram)

Scenario
The customer has the standard AEM Publish and Apache web servers (shown as #1 in the diagram) already setup and there are multiple websites hosted here.
For one new site, there is a need to access the site through a new set of web servers (shown as #2 in the diagram) hosted on a 3rd party cloud.
If you look closely, this is a different-than-regular scenario with 2 web servers in the request path. Usually we have just one web layer load balancing AEM publish instances.

I am able to access the website via LB 1's URL origin-some-domain.com. LB has a valid certificate installed on it. LB accepts requests directly on port 443 and forces all requests to HTTP port 80 requests to port 443.

Problem

Next, I want to hook dispatcher #2 to this LB 1. There is no blocking firewall between 3rd party web servers and the LB.

Both the web servers #2 and Apache #2 do NOT have SSL enabled. I am not sure if SSL is needed on Apache #2 or Dispatcher #2 or both to communicate with LB 1. Please suggest if that is needed and guidelines around how to do it?

I see the following Adobe documentation for Dispatcher with SSL and AEM, I am not sure if that applies to Dispatcher in this scenario too:

https://docs.adobe.com/content/help/en/experience-manager-dispatcher/using/configuring/dispatcher-ss...

My dispatcher config looks like this,

/farms
{
     # First farm entry
    /website
      {
      # Request headers that should be forwarded to the remote server.
   /clientheaders
      {
          "*"
      }

# Hostname globbing for farm selection (virtual domain addressing)
/virtualhosts
   {
      "*"
   }
# The load will be balanced among these render instances
/renders
  {
    /rend01
    {
        /hostname "website-domain.com"
       /port "443"
       /timeout "00"
    }
}

...

...

..

When i look at the logs, I see the following errors,

[Wed Jul 22 03:58:30 2020] [I] [pid 24179 (tid 139991841765120)] "HEAD /content/path-to-content/home.html" 502 - 889ms [website/rend01]
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Connected to backend rend01 (website-domain.com:443)
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Adding request header: host
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Adding request header: Via
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Adding request header: X-Forwarded-For
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Adding request header: Server-Agent
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Unable to parse response: premature end in HTTP response (Reading from remote peer failed: Connection reset by peer), state = 0
[Wed Jul 22 03:58:30 2020] [W] [pid 24179 (tid 139991673976576)] Failed parsing response: premature end in HTTP response (Reading from remote peer failed: Connection reset by peer).
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] initializing retry, closing socket
[Wed Jul 22 03:58:30 2020] [D] [pid 24179 (tid 139991673976576)] Creating new socket: website-domain.com.cn:443
[Wed Jul 22 03:58:31 2020] [D] [pid 24179 (tid 139991799801600)] Found farm website for website-domain.com
[Wed Jul 22 03:58:31 2020] [D] [pid 24179 (tid 139991799801600)] checking [/content/path-to-content/home.html]
[Wed Jul 22 03:58:31 2020] [D] [pid 24179 (tid 139991799801600)] cachefile does not exist: /app/aem/content/path-to-content/home.html
[Wed Jul 22 03:58:31 2020] [D] [pid 24179 (tid 139991799801600)] cache-action for [/content/path-to-content/home.html]: NONE
[Wed Jul 22 03:58:31 2020] [D] [pid 24179 (tid 139991799801600)] Creating new socket: website-domain.com.cn:443

Has anyone come across a similar setup? Can you please guide here what am I missing here/doing wrong?

Regards,

Preetpal

[Albin_Issac]

Why there are two layers of dispatcher?

Can't you point the DNS directly to the #1 load balancer?

The dispatcher cant point to another load balancer or dispatcher, based on your scenario the Apache ProxyPass configurations(Reverse Proxy Configuration) can be used in #2 Apache server to send the request for specific URL's to #1 load balancer URL

Read more about Apache ProxyPass here - https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

Regards

Albin I

www.albinsblog.com

[berliant · Accepted solution]

I agree with albinissac 

Two subsequent dispatchers are not a proper configuration. If you have multiple sites that need to be accessed as different sites you can use a single Apache.

Please review KB at https://docs.adobe.com/content/help/en/experience-manager-dispatcher/using/configuring/dispatcher-do..., it describes that scenario in details 

[Preetpal_Bindra]

@berliant @  @Albin_Issac 

You are absolutely correct! This does not look ideal, however there are multiple factors that lead us to this setup. There are few more options on the table that we are considering, however, I wanted to explore this option before we simply drop it.

I have been able to connect 2 dispatchers in a row and the traffic flows as expected #2 > #1 > Publishers and back. So, that is solved and I have tested it. It works over HTTP.

Dispatcher #2 > LB (HTTP) > Dispatcher #1 > Publishers

What I am not sure about, and need help to understand, is how to connect dispatcher #2 to a LB over HTTPS? I am willing to try options to make this work.

From what I have read so far, such a setup may need, (https://docs.adobe.com/content/help/en/experience-manager-dispatcher/using/configuring/dispatcher-ss...)

1. the ssl version of the dispatcher,

2. could need a virtual host file for port 443,

3. Same certificate installed both on dispatcher and the LB.

4. apache.conf udpates for the dispatcher module.

As per Adobe documentation, the above setup is recommended for Dispatcher > AEM (HTTPS). I am not sure if the same setup could work for Dispatcher > LB (HTTPS) too. What are your suggestions regarding that?

Regards,

Preetpal

[berliant]

Two subsequent dispatchers are not even a supported topology. You should not consider this scenario.