difference between invalidate and statfileslevel

[JakeCham]

Hi Team ,

Can you explain difference between two properties named "invalidate" and "statfileslevel" in AEM cache section please ..

[KannanC · Accepted solution]

Hi @JakeCham 

The /invalidate property defines the documents that are automatically invalidated when content is updated. With automatic invalidation, Dispatcher does not delete cached files after a content update, but checks their validity when they are next requested. Documents in the cache that are not auto-invalidated remain in the cache until a content update explicitly deletes them.

Automatic invalidation is typically used for HTML pages. HTML pages often contain links to other pages, making it difficult to determine whether a content update affects a page. To make sure that all relevant pages are invalidated when content is updated, automatically invalidate all HTML pages. 

configuration example looks as follows

/invalidate
  {
   /0000 { /glob "*" /type "deny" }
   /0001 { /glob "*.html" /type "allow" }
   /0002 { /glob "*.zip" /type "allow" }
   /0003 { /glob "*.pdf" /type "allow" }
  }

Use the /statfileslevel property to invalidate cached files according to their path

Dispatcher creates .statfiles in each folder from the docroot folder to the level that you specify. The docroot folder is level 0.

Files are invalidated by touching the .stat file. The .stat file’s last modification date is compared to the last modification date of a cached document. The document is re-fetched if the .stat file is newer.

When a file at a certain level is invalidated, all .stat files from the docroot to the level of the invalidated file or the configured statsfilevel (whichever is smaller) are touched.

Reference : https://www.aemrules.com/2020/04/dispactherhacks-statfile.html 

                    https://experienceleague.adobe.com/docs/experience-manager-dispatcher/using/configuring/dispatcher-c...l

[Jvenkat]

Hi @JakeCham 

1. invalidate: This property in dispatcher.any controls whether or not Dispatcher should automatically drop (invalidate) related cached files when an incoming request modifies the content on a publish instance. When content is published, an HTTP POST request is sent to Dispatcher which then invalidates the relevant cached files. If invalidate is set to 0, Dispatcher will not invalidate the files automatically and they have to be manually invalidated or re-validated. If you set invalidate to 1, Dispatcher will automatically invalidate the cache when it receives a POST request. In most scenarios, you should set invalidate to 1 to ensure that stale content is regularly cleared from the cache.

2. statfileslevel: This property in dispatcher.any controls the depth in the cache hierarchy at which Dispatcher creates .stat files. .stat files provide information about the last invalidation of the cache. When Dispatcher receives a cache invalidation command, it updates the relevant .stat file. When a user requests a file, Dispatcher checks the timestamp of corresponding .stat file. If the .stat file is newer than the cached file, Dispatcher will re-fetch the content from publish instance, hence ensuring the user gets the most recent version of the content. The statfileslevel parameter thus controls the granularity at which you track changes in your cache. A lower value will give you more granular control, while a higher value has less overhead but gives a less precise control.

Thanks, 

Venkat

[Imran__Khan]

@JakeCham In Adobe Experience Manager (AEM), invalidate and statfileslevel are two different concepts related to caching and performance optimization.

Invalidate:invalidate is a command or action used to clear the cached content in AEM. When you invalidate a resource or a page in AEM, you're essentially telling the system to remove the cached version of that resource/page. This can be useful when you've made changes to content or configurations that you want to reflect immediately without waiting for the cache to expire naturally.

Statfileslevel:statfileslevel is a configuration parameter used in AEM dispatcher setups. AEM dispatcher is a caching and load balancing tool that sits in front of AEM instances to improve performance and reliability. statfileslevel determines the level at which the dispatcher checks for changes in the cached files. It's essentially a mechanism to trigger cache invalidation based on file system metadata changes. Different values of statfileslevel specify how deep in the directory structure the dispatcher will check for changes.

For example, setting statfileslevel to 0 means the dispatcher will only check the existence of files (but not their contents), while setting it to a higher value means it will recursively check files and directories deeper in the hierarchy.

In summary, while both invalidate and statfileslevel are related to cache management in AEM, they operate at different levels and serve different purposes. Invalidate directly clears cached content, whereas statfileslevel is a configuration parameter used in AEM dispatcher setups to determine how file system changes trigger cache invalidation.