Expand my Community achievements bar.

Dive into Adobe Summit 2024! Explore curated list of AEM sessions & labs, register, connect with experts, ask questions, engage, and share insights. Don't miss the excitement.
SOLVED

Dispatcher Filter Configuration

Avatar

Level 2

Hello,

 

Well, I need to configure dispatcher so I can access a get servlet I've created. It works in author and publish instances. I've also already configured the dispatcher to run on port 8080 using docker and I'm able to access published content via port 8080. Nevertheless, when I try to access my servlet via this link: localhost:8080/bin/random-number it gives me this page with a 404 not found error when I check the browser console: image.png

The linux command I'm using to run dispatcher on my project directory through docker is

 

sudo ~/aem-sdk/dispatcher/dispatcher-sdk-2.0.174/bin/docker_run.sh /home/webjump/estágio/trilha-04/aem-wknd-3.2.0/dispatcher/src host.docker.internal:4503 8080

 

My dispatcher filter configuration files are

1 - default_filters.any:

 

 

#
# This is the default filter ACL specifying what requests are handled by the dispatcher.
#
# DO NOT EDIT this file, your changes will have no impact on your deployment.
#
# Instead modify filters.any.
#

# deny everything and allow specific entries
# Start with everything blocked as a safeguard and open things customers need and what's safe OOTB
/0001 { /type "deny"  /url "*" }

# Open consoles if this isn't a production environment by uncommenting the next few lines
# /002 { /type "allow" /url "/crx/*"    }  # allow content repository
# /003 { /type "allow" /url "/system/*" }  # allow OSGi console

# allow non-public content directories if this isn't a production environment by uncommenting the next few lines
# /004 { /type "allow" /url "/apps/*"   }  # allow apps access
/005 { /type "allow" /url "/bin/*"    }  # allow bin path access

# Allow specific URLs
/006-allow-random-number { /type "allow" /url "/bin/random-number" }

# This rule allows content to be access
/0010 { /type "allow" /extension '(css|eot|gif|ico|jpeg|jpg|js|gif|pdf|png|svg|swf|ttf|woff|woff2|html|mp4|mov|m4v)' /path "/content/*" }  # disable this rule to allow mapped content only

# Enable specific mime types in non-public content directories
/0011 { /type "allow" /method "GET" /extension '(css|eot|gif|ico|jpeg|jpg|js|gif|png|svg|swf|ttf|woff|woff2)' }

# Enable clientlibs proxy servlet
/0012 { /type "allow" /method "GET" /url "/etc.clientlibs/*" }

# Enable basic features
/0013 { /type "allow" /method "GET" /url '/libs/granite/csrf/token.json' /extension 'json' } # AEM provides a framework aimed at preventing Cross-Site Request Forgery attacks
/0014 { /type "allow" /method "POST" /url "/content/*.form.html" }  # allow POSTs to form selectors under content

/0015 { /type "allow" /method "GET" /path "/libs/cq/personalization" }  # enable personalization
/0016 { /type "allow" /method "POST" /path "/content/*.commerce.cart.json" }  # allow POSTs to update the shopping cart

# Deny content grabbing for greedy queries and prevent un-intended self DOS attacks
/0017 { /type "deny" /selectors '(feed|rss|pages|languages|blueprint|infinity|tidy|sysview|docview|query|[0-9-]+|jcr:content)' /extension '(json|xml|html|feed)' }

# Deny authoring query params
/0018 { /type "deny" /method "GET" /query "debug=*" }
/0019 { /type "deny" /method "GET" /query "wcmmode=*" }

# Allow current user
/0020 { /type "allow" /url "/libs/granite/security/currentuser.json" }

# Allow index page
/0030 { /type "allow" /url "/index.html" }

# Allow IMS Authentication
/0031 { /type "allow" /method "GET" /url "/callback/j_security_check" }

# AEM Forms specific filters
# to allow AF specific endpoints for prefill, submit and sign
/0032 { /type "allow" /path "/content/forms/af/*" /method "POST" /selectors '(submit|internalsubmit|agreement|signSubmit|prefilldata|save|analyticsconfigparser)' /extension '(jsp|json)' }

# to allow AF specific endpoints for thank you page
/0033 { /type "allow" /path "/content/forms/af/*"  /method "GET" /selectors '(guideThankYouPage|guideAsyncThankYouPage)'  /extension '(html)'}

# to allow AF specific endpoints for lazy loading
/0034 { /type "allow" /path "/content/forms/af/*"  /method "GET" /extension '(jsonhtmlemitter)'}

# to allow fp related functionalities
/0035 { /type "allow" /path "/content/forms/*" /selectors '(fp|attach|draft|dor|api)'  /extension '(html|jsp|json|pdf)' }

# to allow forms access via dam path
/0036 { /type "allow" /path "/content/dam/formsanddocuments/**/jcr:content" /method "GET"}

# to allow invoke service functionality (FDM)
/0037 { /type "allow" /path "/content/forms/*" /selectors '(af)'  /extension '(dermis)' }

# to allow forms portal draft and submissions component operation servlet
/0038 { /type "allow" /path "/content/*" /method "GET" /selectors '(fp)' /extension '(operation)' }

# AEM Screens Filters
# to allow AEM Screens channels selectors
/0050 { /type "allow" /method "GET" /url "/screens/channels.json" }

# to allow AEM Screens Content and selectors
/0051 { /type "allow" /method '(GET|HEAD)' /url "/content/screens/*" }

# AEM Sites Filters
# to allow site30 theme servlet
/0052 { /type "allow" /extension "theme" /path "/content/*" }

# Allow manifest.webmanifest files located in the content
/0053 { /type "allow" /extension "webmanifest" /path "/content/*/manifest" }

# Allow Apache Sling Sitemap selectors: sitemap, sitemap-index, sitemap.any-nested-or-named-sitemap
/0054 { /type "allow" /method "GET" /path "/content/*" /selectors 'sitemap(-index)?' /extension "xml" }

# Allow GraphQL & preflight requests
# GraphQL also supports "GET" requests, if you intend to use "GET" add a rule in filters.any
/0060 { /type "allow" /method '(POST|OPTIONS)' /url "/content/_cq_graphql/*/endpoint.json" }

# GraphQL Persisted Queries & preflight requests
/0061 { /type "allow" /method '(GET|POST|OPTIONS)' /url "/graphql/execute.json*" }

# Allow Forms Document Services requests
/0062 { /type "allow" /method '(GET|POST)' /url "/adobe/forms/*" }

 

 

2 - filters.any

 

 

#
# This file contains the filter ACL, and can be customized.
#
# By default, it includes the default filter ACL.
#

$include "./default_filters.any"

# allow non-public content directories if this isn't a production environment by uncommenting the next few lines
/005 { /type "allow" /url "/bin/*"    }  # allow bin path access

# Allow specific URLs
/006-allow-random-number { /type "allow" /url "/bin/random-number" }

# Allow components JSON model
/0101 { /type "allow" /extension "json" /selectors "model" /path "/content/*" }

# Allow manifest.webmanifest files located in the content
/0102 { /type "allow" /extension "webmanifest" /path "/content/*/manifest" }

# Allow Quick Search
/0110 { /type "allow" /extension "json" /selectors "searchresults" /path "/content/*" }

# Allow j_security check for login
/0111 { /type "allow" /method "POST" /url "/system/sling/login/j_security_check" }

#Allow logout path
/0112 { /type "allow" /method "POST" /url "/system/sling/logout" }

# Allow GraphQL global endpoint & preflight requests
# GraphQL also supports GET requests, if you intend to use GET include it in the rule below
/0164 { /type "allow" /method '(POST|OPTIONS)' /url "/content/graphql/global/endpoint.json" /extension "json"}
/0165 { /type "allow" /method '(POST|OPTIONS)' /url "/content/cq:graphql/wknd/endpoint.json" /extension "json"}

# GraphQL Persisted Queries
/0170 { /type "allow" /method '(POST|OPTIONS)' /url "/graphql/execute.json/*" }
/0171 { /type "allow" /url "/graphql/execute.json/*" }

# Allow to download web manifest for PWA
/0180 { /type "allow" /extension "webmanifest" }

# ContextHub
/0200 { /type "allow" /url "*/contexthub.pagedata.json" }
/0201 { /type "allow" /url "/home/users/*.infinity.json" }

 

 

What am I missing? Are there any other files that I need to alter? How can I know where is the problem?

 

Update: this is what I'm getting with the debug mode

 

[Tue Nov 28 15:01:00.932224 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Found farm publishfarm for localhost:8080
[Tue Nov 28 15:01:00.932247 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] checking [/bin/random-number]
[Tue Nov 28 15:01:00.932258 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] request URL has no extension: /bin/random-number
[Tue Nov 28 15:01:00.932262 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] cache-action for [/bin/random-number]: NONE
[Tue Nov 28 15:01:00.932313 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Filter rejects: GET /bin/random-number HTTP/1.1
[Tue Nov 28 15:01:00.932502 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] Found farm publishfarm for localhost:8080
[Tue Nov 28 15:01:00.932513 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] checking [/content/wknd/us/en/errors/404.html]
[Tue Nov 28 15:01:00.932551 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] never flushed [/mnt/var/www/html/content/wknd/.stat] -> use cache [/mnt/var/www/html/content/wknd/us/en/errors/404.html]
[Tue Nov 28 15:01:00.932564 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] cache-action for [/content/wknd/us/en/errors/404.html]: DELIVER
[Tue Nov 28 15:01:00.932568 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] request declined
[Tue Nov 28 15:01:00.932893 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[Content-Type] = "text/html;charset=utf-8"
[Tue Nov 28 15:01:00.932906 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[X-Content-Type-Options] = "nosniff"
[Tue Nov 28 15:01:00.932921 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[Age] = "2937875"
[Tue Nov 28 15:01:01.002473 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Found farm publishfarm for localhost:8080, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002487 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] checking [/etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub], referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002492 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] request URL has no extension: /etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002496 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] cache-action for [/etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub]: NONE, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002626 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Creating new connection: 192.168.0.11:4503, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002759 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Connected to backend 0 (192.168.0.11:4503), referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039008 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.status = 200, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039029 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[Date] = "Tue, 28 Nov 2023 15:01:01 GMT", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039036 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[X-Content-Type-Options] = "nosniff", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039041 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[X-Frame-Options] = "SAMEORIGIN", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039046 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[ETag] = ""67400b98d9aa58b9970dc399607ec85102e979a6"", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039051 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[Content-Type] = "application/javascript; charset=utf-8", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.049161 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] No socket reuse: backend asks to close connection, referer: http://localhost:8080/bin/random-number

 

 Now I can see that the filter is rejecting the request, even though I've modified filters.any to allow it. Did I modify the wrong file? Is the syntax wrong? Is it missing something?

1 Accepted Solution

Avatar

Correct answer by
Community Advisor

Also, in your case Dispatcher won’t cache the response because there is no file extension in the url.

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/cache-servlet-response/m-p...

View solution in original post

15 Replies

Avatar

Community Advisor

What entry do you see in dispatcher log? Can you start the dispatcher in debug mode to check what happening for this request ?

Avatar

Level 1

Actually, I don't know how to do that. Where can I find the dispatcher logs? How can I start dispatcher in debug mode?

Avatar

Community Advisor

It seems you are using dispatcher on local using docker for this test so you can update the DISP_LOG_LEVEL located on conf.d/variables/global.vars 

Avatar

Level 2

I thought I've already answered you, I don't know what happened. Maybe my comment wasn't allowed. I've updated the post with the debug logs, but I'll post them here too:

[Tue Nov 28 15:01:00.932224 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Found farm publishfarm for localhost:8080
[Tue Nov 28 15:01:00.932247 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] checking [/bin/random-number]
[Tue Nov 28 15:01:00.932258 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] request URL has no extension: /bin/random-number
[Tue Nov 28 15:01:00.932262 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] cache-action for [/bin/random-number]: NONE
[Tue Nov 28 15:01:00.932313 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Filter rejects: GET /bin/random-number HTTP/1.1
[Tue Nov 28 15:01:00.932502 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] Found farm publishfarm for localhost:8080
[Tue Nov 28 15:01:00.932513 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] checking [/content/wknd/us/en/errors/404.html]
[Tue Nov 28 15:01:00.932551 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] never flushed [/mnt/var/www/html/content/wknd/.stat] -> use cache [/mnt/var/www/html/content/wknd/us/en/errors/404.html]
[Tue Nov 28 15:01:00.932564 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] cache-action for [/content/wknd/us/en/errors/404.html]: DELIVER
[Tue Nov 28 15:01:00.932568 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] request declined
[Tue Nov 28 15:01:00.932893 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[Content-Type] = "text/html;charset=utf-8"
[Tue Nov 28 15:01:00.932906 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[X-Content-Type-Options] = "nosniff"
[Tue Nov 28 15:01:00.932921 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:0] response.headers[Age] = "2937875"
[Tue Nov 28 15:01:01.002473 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Found farm publishfarm for localhost:8080, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002487 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] checking [/etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub], referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002492 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] request URL has no extension: /etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002496 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] cache-action for [/etc/cloudsettings.kernel.js/conf/wknd/settings/cloudsettings/default/contexthub]: NONE, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002626 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Creating new connection: 192.168.0.11:4503, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.002759 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Connected to backend 0 (192.168.0.11:4503), referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039008 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.status = 200, referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039029 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[Date] = "Tue, 28 Nov 2023 15:01:01 GMT", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039036 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[X-Content-Type-Options] = "nosniff", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039041 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[X-Frame-Options] = "SAMEORIGIN", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039046 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[ETag] = ""67400b98d9aa58b9970dc399607ec85102e979a6"", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.039051 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] response.headers[Content-Type] = "application/javascript; charset=utf-8", referer: http://localhost:8080/bin/random-number
[Tue Nov 28 15:01:01.049161 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] No socket reuse: backend asks to close connection, referer: http://localhost:8080/bin/random-number

 I can see now that the filter is rejecting the requests, but why? I've altered the filters.any file in the dispatcher directory. Is it that I'm altering the wrong file? Or the syntax is wrong?

Avatar

Community Advisor
[Tue Nov 28 15:01:00.932313 2023] [dispatcher:debug] [pid 313:tid 139849276103480] [client localhost:33972] Filter rejects: GET /bin/random-number HTTP/1.1

Above log shows that one of filter is rejecting this path. I suspect your filters.any file is not loading in your dispatcher. Changes made on default_filter.any will never be picked up. How are you calling the filter.any file in your dispatcher setup?. Try setting up your custom farm for your site and then call your custom filters.any

Avatar

Level 2

I've found the problem. I've modified the filters.any file at the dispatcher installation folder, but not in the wknd project dispatcher directory. I'm sorry about that, I'm an intern and I'm not having as much help as I should (so I kind of need to figure things out on my own and adobe tutorials can be a little confusing or incomplete).

 

Now I have a second problem: the get request is going through dispatcher as planned, but I need to get the same answer (the same number) with cached content. I'm using the wknd project default configuration. As far as I'm concerned, this line of code on default_rules.any should allow cached content

/0000 {
	/glob "*"
	/type "allow"
}

 

But I'm getting this from dispatcher debug logs:

[dispatcher:debug] [pid 313:tid 140306258733880] [client localhost:35264] cache-action for [/bin/random-number]: NONE

 Any hints about where could be the problem?

 

Avatar

Level 3

May be you are not updating the host (wknd) specific file. 

default_rules.any is probably not applicable to your site.

Simple test will be 

change the rule to

 

/0000 {
	/glob "*"
	/type "deny"
}

 

restart your apache service 

Then browse any other url of your site

Check dispatcher logs for cache-action 

If the rule is effective it shall deny caching other files too 

Avatar

Community Advisor

Thanks for Confirming that you are now able to get the response for your servlet via dispatcher. If you looks the aem dispatcher farm.any in case it might be wknd_farm.any check for something similar inside conf.dispatcher.d/available_farms. So on your site-name_farm.any you can see several sections. /filter section is for filtering the request and to cache rules for caching the response will be inside /rules in /cache section where it might be referring to some file located inside /conf.dispatcher.d/cache/site-name-cache.any

Avatar

Correct answer by
Community Advisor

Also, in your case Dispatcher won’t cache the response because there is no file extension in the url.

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/cache-servlet-response/m-p...

Avatar

Level 2

Thank you! I've managed to solve the problem. I've changed allowAuthorized to 1 instead of 0 and added the .json extension permission to the wknd.farm file inside enabled_farms directory.

# caches also authorized data
		/allowAuthorized "1"

# cache.
		/invalidate {
			/0004 {
				/glob "*.json"
				/type "allow"
			}

 

Avatar

Level 3

You can find dispatcher logs under /etc/httpd/logs directory 

As @DPrakashRaj mention you shall enable debug logging and restart dispatcher to see the error details.

Avatar

Community Advisor

You should not update default filter rules

you must change your won custom file with rule.



Arun Patidar

Avatar

Administrator

@RafaelSp 

Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.



Kautuk Sahni

Avatar

Level 2

It was helpful since now I'm able to use dispatcher in debug mode. I've updated the post with the error logs. Nevertheless, I'm still stuck

Avatar

Level 3

@RafaelSp  in your dispatcher. any file check /filter section. You will probably have  a line which includes a filter file 

example 

/filter
{
$include "publish-filters.any"
}

 

Make sure you are doing the changes into the filter file which is included into the dispatcher.any file.  If you have more than one filter files then probably a filter added latter is taking preference.