Level 2

Solved

dispacher unable to cache the pages

Forum|Forum|9 years ago
May 3, 2016
7 replies
3802 views

Dispatcher is unable to cache the pages on my DR server however similar configuration working on my local.

try to create new cachefile: /var/www/Pub-cache/personal-banking/products/accounts/individual-savings-account.html
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] cache-action for [/personal-banking/products/accounts/individual-savings-account.html]: CREATE
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] connected to backend rend01 (10.24.65.57:4503)
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Host
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: User-Agent
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Accept
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Accept-Language
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Accept-Encoding
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Referer
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Spooling request header: Cookie
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Detected: chunked transfer encoding
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Detected: dispatcher must not cache
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Detected: dispatcher must not cache
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] response.status = 200
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] response.headers[Server] = "Day-Servlet-Engine/4.1.52 "
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] response.headers[Content-Type] = "text/html;charset=utf-8"
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] response.headers[Date] = "Tue, 03 May 2016 11:27:28 GMT"
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] response.headers[Cache-Control] = "no-cache"
[Tue May 03 16:42:52 2016] [D] [18412(140522128189408)] Content length: -1, written: 174875
[Tue May 03 16:42:52 2016] [I] [18412(140522128189408)] "GET /personal-banking/products/accounts/individual-savings-account.html" 200 174822 418ms

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by joerghoh

The dispatcher caches under these conditions:

It's a get GET request: You have a GET request.
The request does not have a query string. Your request does not have a query string.
The request is not filtere. Obviously your request gets through to AEM, so it's not filtered.
The request is covered by a cache rule; this statement "cache-action for [/personal-banking/products/accounts/individual-savings-account.html]: CREATE" indicates that the dispatcher would cache the file.
The page is considered cacheable by AEM, so no "no-cache" header is set. This is not the case, see the message "[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Detected: dispatcher must not cache"

So your AEM instance instructs all intermediate caches (including the dispatcher) not to cache this response. If I extrapolate the content of the page from its name, it is better if it is not cached...

Jörg

Peter_Puzanovs

Community Advisor

Does your apache instance has permissions to save the file in that path?

What are the rules in dispatcher.conf and .any file for this domain/paths?

Regards,

Peter

smacdonald2008

Level 10

Also in addition, please post log messages. Thxs!

A

askdctm

Level 9

Hi,

Just a thought.

In the entries above, I can see "no-cache". Doesn't it imply that this particular request should not be cached.

D

Deepaks007Author

Level 2

# Each farm configures a set of load balanced renders (i.e. remote servers)
/farms
{
# First farm entry
/website
    {
    # Request headers that should be forwarded to the remote server.
    /clientheaders
      {
      # Forward all request headers that are end-to-end. If you want
      # to forward a specific set of headers, you'll have to list
      # them here.
      "*"

       }

    # Hostname globbing for farm selection (virtual domain addressing)
    /virtualhosts
      {
      # Entries will be compared against the "Host" request header
      # and an optional request URL prefix.
      #
      # Examples:
      #
      #   www.company.com
      #   intranet.*
      #   myhost:8888/mysite
      "*"
      }

    # The load will be balanced among these render instances
    /renders
      {
      /rend01
        {
        # Hostname or IP of the render
#        /hostname "10.24.121.109"
        /hostname "10.24.65.57"
        # Port of the render
        /port "4503"
        # Connect timeout in milliseconds, 0 to wait indefinitely
        # /timeout "0"
        }
      }

    # The filter section defines the requests that should be handled by the dispatcher.
    # The globs will be compared against the request line, e.g. "GET /index.html HTTP/1.1".
    /filter
      {
      # Deny everything first and then allow specific entries
      /0001 { /type "deny" /glob "*" }

      # Open consoles
#     /0011 { /type "allow" /glob "* /admin/*" } # allow servlet engine admin
#     /0012 { /type "allow" /glob "* /crx/*"    } # allow content repository
#     /0013 { /type "allow" /glob "* /system/*" } # allow OSGi console

      # Allow non-public content directories
#     /0021 { /type "allow" /glob "* /apps/*"   } # allow apps access
      /0022 { /type "allow" /glob "* /bin/*"    }
      /0023 { /type "allow" /glob "* /content*" } # disable this rule to allow mapped content only
      /0024 { /type "allow" /glob "* /*" }

      /00232 { /type "allow" /glob "* /etc/designs*" }
      /00233 { /type "allow" /glob "* /etc/scripts*" }
      /00234 { /type "allow" /glob "* /robots.txt*" }

#      /00235 { /type "allow" /glob "* /etc/client*" }
#      /00236 { /type "deny" /glob "* /libs/*" }

# /0025 { /type "allow" /glob "* /libs/*" }
# /0026 { /type "deny" /glob "* /libs/shindig/proxy*" } # if you enable /libs close access to proxy

#     /0027 { /type "allow" /glob "* /home/*"   }
#     /0028 { /type "allow" /glob "* /tmp/*"    }
#     /0029 { /type "allow" /glob "* /var/*"    }

      # Enable specific mime types in non-public content directories
      /0041 { /type "allow" /glob "* *.css *"   } # enable css
      /0042 { /type "allow" /glob "* *.gif *"   } # enable gifs
      /0043 { /type "allow" /glob "* *.ico *"   } # enable icos
      /0044 { /type "allow" /glob "* *.js *"    } # enable javascript
      /0045 { /type "allow" /glob "* *.png *"   } # enable png
      /0046 { /type "allow" /glob "* *.swf *"   } # enable flash
      /0047 { /type "allow" /glob "* *.woff *"   } # enable fonts

      # Enable features
      /0061 { /type "allow" /glob "POST /content/[.]*.form.html" } # allow POSTs to form selectors under content
      /0062 { /type "allow" /glob "* /libs/cq/personalization/*" } # enable personalization

      # Deny content grabbing
      /0081 { /type "deny" /glob "GET *.infinity.json*" }
      /0082 { /type "deny" /glob "GET *.tidy.json*"     }
      /0083 { /type "deny" /glob "GET *.sysview.xml*"   }
      /0084 { /type "deny" /glob "GET *.docview.json*" }
      /0085 { /type "deny" /glob "GET *.docview.xml*" }

/0086 { /type "deny" /glob "GET *.*[0-9].json*" }
# /0087 { /type "allow" /glob "GET *.1.json*" } # allow one-level json requests

      # Deny query
      /0090 { /type "deny" /glob "* *.query.json*" }
      }

    # The cache section regulates what responses will be cached and where.
    /cache
      {
      # The docroot must be equal to the document root of the webserver. The
      # dispatcher will store files relative to this directory and subsequent
      # requests may be "declined" by the dispatcher, allowing the webserver
      # to deliver them just like static files.
      /docroot "/var/www/Pub-cache"

      # Sets the level upto which files named ".stat" will be created in the
      # document root of the webserver. When an activation request for some
      # page is received, only files within the same subtree are affected
      # by the invalidation.
       /statfileslevel "3"

      # Flag indicating whether to cache responses to requests that contain
      # authorization information.
      /allowAuthorized "1"

      # Flag indicating whether the dispatcher should serve stale content if
      # no remote server is available.
      /serveStaleOnError "1"

      # The rules section defines what responses should be cached based on
      # the requested URL. Please note that only the following requests can
      # lead to cacheable responses:
      #
      # - HTTP method is GET
      # - URL has an extension
      # - Request has no query string
      # - Request has no "Authorization" header (unless allowAuthorized is 1)
      /rules
        {
        /0000
          {
          # the globbing pattern to be compared against the url
          # example: *             -> everything
          #        : /foo/bar.*    -> only the /foo/bar documents
          #        : /foo/bar/*    -> all pages below /foo/bar
          #        : /foo/bar[./]* -> all pages below and /foo/bar itself
          #        : *.html        -> all .html files
          /glob "*"
          /type "allow"
          }
#        /0001 { /glob "*" /type "allow" /compress "gzip" }
        }

      # The invalidate section defines the pages that are "invalidated" after
      # any activation. Please note that the activated page itself and all
      # related documents are flushed on an modification. For example: if the
      # page /foo/bar is activated, all /foo/bar.* files are removed from the
      # cache.
      /invalidate
        {
        /0000
          {
          /glob "*"
          /type "allow"
          }
        /0001
          {
          # Consider all HTML files stale after an activation.
          /glob "/content/home/locate-us.html"
          /type "deny"
          }
        }

      # The allowedClients section restricts the client IP addresses that are
      # allowed to issue activation requests.
      /allowedClients
        {
        # Uncomment the following to restrict activation requests to originate
        # from "localhost" only.
        #
        #/0000
        # {
        # /glob "*"
        # /type "deny"
        # }
        #/0001
        # {
        # /glob "127.0.0.1"
        # /type "allow"
        # }
        }

      # The ignoreUrlParams section contains query string parameter names that
      # should be ignored when determining whether some request's output can be
      # cached or delivered from cache.
      #
      # In this example configuration, the "q" parameter will be ignored.
      /ignoreUrlParams
        {
      # /0001 { /glob "*" /type "deny" }
      # /0002 { /glob "q" /type "allow" }
        /0003 { /glob "_" /type "allow" }
        /0004 { /glob "view" /type "allow" }
        }

}

    # The statistics sections dictates how the load should be balanced among the
    # renders according to the media-type.
    /statistics
      {
      /categories
        {
        /html
          {
          /glob "*.html"
          }
        /others
          {
          /glob "*"
          }
        }
      }
    }
}

D

Deepaks007Author

Level 2

Hi Scott, please refer log

Tue May 03 18:19:09 2016] [I] [19238(140429846185952)] "GET /content/home/personal-banking.html" 200 261235 402ms
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Found farm website for www.indusind.com
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] checking [/personal-banking/products/accounts/individual-savings-account.html]
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] cachefile does not exist or is a directory: /var/www/Pub-cache/personal-banking/products/accounts/individual-savings-account.html
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] try to create new cachefile: /var/www/Pub-cache/personal-banking/products/accounts/individual-savings-account.html
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] cache-action for [/personal-banking/products/accounts/individual-savings-account.html]: CREATE
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] connected to backend rend01 (10.24.65.57:4503)
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: referer
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: user-agent
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: accept-encoding
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: accept-language
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: accept
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: host
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Spooling request header: cookie
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Detected: chunked transfer encoding
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Detected: dispatcher must not cache
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Detected: dispatcher must not cache
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] response.status = 200
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] response.headers[Server] = "Day-Servlet-Engine/4.1.52 "
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] response.headers[Content-Type] = "text/html;charset=utf-8"
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] response.headers[Date] = "Tue, 03 May 2016 13:03:53 GMT"
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] response.headers[Cache-Control] = "no-cache"
[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Content length: -1, written: 174875
[Tue May 03 18:19:17 2016] [I] [19241(140429846185952)] "GET /personal-banking/products/accounts/individual-savings-account.html" 200 174822 426ms
[Tue May 03 19:38:52 2016] [D] [19241(140429846185952)] Found farm website for 10.24.25.172
[Tue May 03 19:38:52 2016] [D] [19241(140429846185952)] checking [/content/Indulge/dispatcher/invalidate.cache]
[Tue May 03 19:38:52 2016] [I] [19241(140429846185952)] Activation detected: action=Activate [/content/usergenerated/content/rateid/persregularrecurringdeposit/ratecount]
[Tue May 03 19:38:52 2016] [I] [19241(140429846185952)] Touched /var/www/Pub-cache/.stat
[Tue May 03 19:38:52 2016] [I] [19241(140429846185952)] Touched /var/www/Pub-cache/content/.stat
[Tue May 03 19:38:52 2016] [D] [19241(140429846185952)] response.headers[Server] = "Communique/2.6.3 (build 5221)"
[Tue May 03 19:38:52 2016] [D] [19241(140429846185952)] cache flushed
[Tue May 03 19:38:52 2016] [I] [19241(140429846185952)] "GET /content/Indulge/dispatcher/invalidate.cache" 0 13 0ms
[Tue May 03 20:14:26 2016] [D] [19240(140429846185952)] Found farm website for 10.24.25.172
[Tue May 03 20:14:26 2016] [D] [19240(140429846185952)] checking [/content/Indulge/dispatcher/invalidate.cache]
[Tue May 03 20:14:26 2016] [I] [19240(140429846185952)] Activation detected: action=Activate [/content/usergenerated/content/rateid/persprivateequity/ratecount]
[Tue May 03 20:14:26 2016] [I] [19240(140429846185952)] Touched /var/www/Pub-cache/.stat
[Tue May 03 20:14:26 2016] [I] [19240(140429846185952)] Touched /var/www/Pub-cache/content/.stat
[Tue May 03 20:14:26 2016] [D] [19240(140429846185952)] response.headers[Server] = "Communique/2.6.3 (build 5221)"
[Tue May 03 20:14:26 2016] [D] [19240(140429846185952)] cache flushed
[Tue May 03 20:14:26 2016] [I] [19240(140429846185952)] "GET /content/Indulge/dispatcher/invalidate.cache" 0 13 0ms

Peter_Puzanovs

Community Advisor

Hi,

from the config it looks OK.

1) Can you try to enable on pre-production(staging) environment

/filter
      {
      # Deny everything first and then allow specific entries
      /0001 { /type "deny" /glob "*" }

to allow instead of deny. Just to check whether it would make any difference?

2) Can you check in code, whether you are setting ("Dispatcher", "no-cache") flag somewhere in your code, if it is there then change it?

3) Check whether current user with which you are running Dispatcher has right to create content under that directory?

Regards,

Peter

joerghoh

Accepted solution

Adobe Employee

The dispatcher caches under these conditions:

It's a get GET request: You have a GET request.
The request does not have a query string. Your request does not have a query string.
The request is not filtere. Obviously your request gets through to AEM, so it's not filtered.
The request is covered by a cache rule; this statement "cache-action for [/personal-banking/products/accounts/individual-savings-account.html]: CREATE" indicates that the dispatcher would cache the file.
The page is considered cacheable by AEM, so no "no-cache" header is set. This is not the case, see the message "[Tue May 03 18:19:17 2016] [D] [19241(140429846185952)] Detected: dispatcher must not cache"

So your AEM instance instructs all intermediate caches (including the dispatcher) not to cache this response. If I extrapolate the content of the page from its name, it is better if it is not cached...

Jörg

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded