Caching on AEM Cloud publish adobeaemcloud.com domain

[_CL]

Hey guys,

I am having an issue when I try to access the publish domain https://publish-p00001-e000002.adobeaemcloud.com/content/appName/en/shopping-cart.html

The changes pushed from the author are only visible in publish domain URL when I try to access through the query parameters.

Checking the logs I see the below -

1. aemdispatcher logs - 
   [27/Nov/2024:07:06:44 +0000] [I] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] "GET /content/appName/en/shopping-cart.html" - 11ms [publishfarm-marks/-] [actionhit] publish-p00001-e00002.adobeaemcloud.com
   The request with no query params is being served from dispatcher even after having the cache cleared.
   [27/Nov/2024:07:07:15 +0000] [I] [cm-p00001-e00002-e00002-publish-749967f856-nbzdf] "GET /content/appName/en/shopping-cart.html?nocache=trueCL" 200 326ms [publishfarm-marks/0] [actionnone] publish-p00001-e00002.adobeaemcloud.com
   
   
2. httpd error logs -
   Wed Nov 27 07:06:44.810984 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] checking [/content/appName/en/shopping-cart.html]
   Wed Nov 27 07:06:44.811002 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] never flushed [/mnt/var/www/html/content/appName/.stat] -> use cache [/mnt/var/www/html/content/appName/en/shopping-cart.html]
   Wed Nov 27 07:06:44.811007 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] cache-action for [/content/appName/en/shopping-cart.html]: DELIVER
   Wed Nov 27 07:06:44.811009 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] request declined
   Wed Nov 27 07:06:44.811058 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] response.headers[Content-Type] = "text/html;charset=utf-8"
   Wed Nov 27 07:06:44.811064 2024 [dispatcher:debug] [pid 627:tid 764] [cm-p00001-e00002-aem-publish-749967f856-nbzdf] response.headers[X-Content-Type-Options] = "nosniff"

Why would the logs show the /content/appName/.stat as never flushed when we flushed a child page underneath it?

Is it because of the statfiles level? Its currently set to "2". Do we need to increase it to 3 or 4? How to determine that based on the current node hierarchy?

/statfileslevel "2"

[daniel-strmecki]

Hi @_CL,

There is definietly a problem with your Dispatcher cache not being flushed. Can you confirm that you defined the statfilelevel in the invalidation.farm? As that is the farm used for Dispatcher flush/invalidation requests.

The statfilelevel starts from your DOCROOT, so it looks fine. In the aemdispatcher logs, you should see a HIT or MISS for each request you make. That is how you know if the HTML is served from Dispatcher cache or not.

I suggest you look at the logs of your cache invalidation requests. Also, I suggest you debug the issue on a local Dispatcher instance.

Good luck,

Daniel

[_CL]

hey @daniel-strmecki - I do see hit and miss requests - the requests with query params are reported as [actionhit] while those without are reported as [actionmiss]

I am getting validator issues in my current dispatcher folder - mostly symlinks are not present at enabled vhosts - they are an exact replica of the available vhosts.

[daniel-strmecki]

Hi @_CL,

it should be vice versa, requests with query params should be a MISS. Here is a shell script for you to fix symlinks:

cd ..
project_root=$(pwd)
cd dispatcher/src/conf.d/enabled_vhosts
enabled_hosts=$(ls *.vhost)
for host in $enabled_hosts; do
	rm $host
	ln -s ../available_vhosts/$host $host
done

cd $project_root
cd dispatcher/src/conf.dispatcher.d/enabled_farms
enabled_farms=$(ls *.farm)
for farm in $enabled_farms; do
	rm $farm
	ln -s ../available_farms/$farm $farm
done

Hope this helps,

Daniel

[aanchal-sikka]

@_CL 

Symlinks are not needed which flexible Mode of AEM Dispatcher. Details available on: https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/implementing/con... 

Please verify, if opt-in/USE_SOURCES_DIRECTLY is configure in dispatcher. If yes, symlinks are not needed.

-------

For cached URL with query params, please review if ignoreUrlParams  is configured. https://experienceleague.adobe.com/en/docs/experience-manager-dispatcher/using/configuring/dispatche... 

[aanchal-sikka]

@_CL 

Please enable debug logs on the dispatcher. Publish the child page and validate dispatcher logs. The details on how the logs should look like are available on link

For reference, copying the relevant logs here as well:

With statfilelevel 2, the .stat file would have been created on:

docroot

/content

/content/appName

And any publish on Child-page should touch all .stat files along this path. For better understanding, refer to link

[_CL]

thanks for replying @aanchal-sikka , however referring to your screenshot I see the logs "Touched /mnt/var.." logged as Info and not Debug.
We have already defined the log level as debug and still not getting the logs you mentioned -

Define DISP_LOG_LEVEL debug