AEM 6.5 HEAD Requests

[crisr1]

AEM 6.5 seems to be making regular requests such as the one below; as found in access.log from a publish instance

"HEAD / HTTP/1.1" 200 - "-" "Apache-HttpClient/4.5.4 (Java/1.8.0_251)"

The polling fails constantly on author instances, and the error rate shows up in Java reporting such as New Relic. According to NR, the error is 401 on author instances. The errors reported in New Relic have identifiers (qtp631235405-15166) that match the log below.

*INFO* [qtp631235405-15166] org.apache.sling.auth.core.impl.SlingAuthenticator getAnonymousResolver: Anonymous access not allowed by configuration - requesting credentials

What is generating these HTTP HEAD requests, and is there a way to turn off whatever is making these requests?

Thanks for your comments and question.

[Jörg_Hoh]

Are you really sure that these requests come from AEM itself? Have you ruled out all possibilities that it can come in via dispatcher? What's the interval in which these queries show up?

It doesn't make sense for AEM to send such a request to itself, and I haven't seen something like this yet. At least there's nothing ootb which does that.

[crisr1]

I don't think it's coming through the dispatcher. There are some head requests in the disp, but at a much lower interval and with a different user agent signature

Monitoring the publisher, a few requests comes every 30s

tail -f /mnt/crx/publish/crx-quickstart/logs/access.log | grep --line-buffered HEAD

"HEAD / HTTP/1.1" 200 - "-" "Apache-HttpClient/4.5.4 (Java/1.8.0_251)"

Monitor the dispatcher logs shows a different HEAD request; interval about ~130s

tail -f /mnt/var/log/httpd/access_log | grep --line-buffered HEAD

"HEAD / HTTP/1.1" 200 - "-" "Mozilla/5.0 (X11; Linux x86_64)...

I think there's a health check executed regularly with a signature like org.apache.sling.commons.scheduler.impl.NonParallelQuartzJobExecutor.execute() that's causing it, but probably I'll never know for sure.

The dispatcher does get some HEAD requests, but the browser agent is different Mozilla/5.0 (X11; Linux x86_64), not Apache-HttpClient/4.5.4. The dispatcher does shows some other "health check" requests that come at a regular interval from the same client, but these route externally, I know how to configure those and disable them.

[Asutosh_Jena_ · Accepted solution]

Hi @crisr1 

Could you please provide some more information here. I personally have never seen this type of request on AEM publish instances.
Suspect it might be something that will be coming from dispatcher but will be able to look further more once you provide some more information.

Thanks!

[crisr1]

It is something new I found in AEM 6.5.7 author and publisher instances. It's not present in our previous AEM 6.4.8.3 environments. The user agent mentioned in the HEAD request mentioned Apache-HttpClient/4.5.4 (Java/1.8.0_251), so I doubt that it is originating from the dispatcher. But at the same time, I do not see the same activity in localhost 6.5 instances.

[Jörg_Hoh]

It's not necesarily coming from the dispatcher (as in "the dispatcher sends request to AEM"), but rather that the dispatcher forwards request from somewhere else to AEM. In the AEM access.log there should be also an IP address logged, if this is either 127.0.0.1 or "::1" it's an indication that it's coming from the same machine (and then potentially AEM).

[crisr1]

Thanks for the comments. On AEM publishers access.log, the IP is the same as the publisher instance internal IP (10...)

[Jörg_Hoh]

that means, that this request is originating from the publish, but it's addressing the instance via it's external IP address, and not via "localhost" (which is what I would do if I want to access specifically the same instance). Can you check the logs if the linkchecker is trying to validate that IP address (10.)?