Old JS and CSS served under new name after deployment

[Alexander_Prikhodko]

When a package with JS and CSS code is built and deployed, and then a page requested from the website, it contains links to JS and CSS files processed by AEM (with additional name parts and possibly minified), e.g.: <script src="/etc.clientlibs/foundation/clientlibs/shared.min.9f3716b2c473fd53ab31b1caa53131a8.js">

Generally, when a new version of the package is installed, new JS and CSS file names are generated; and, when they are requested, the new code is served. However, from time to time we notice that, right after deployment, pages already contain links to files with new names but those links serve old code, which often breaks important functionality, as old code is invalid for the new markup.

These invalid JS and CSS files are then cached in visitors' browsers, and even when Publishers start serving the new code, it does not help, as the scripts' names are the same and client browsers get them from cache. The only way to fix it is to deploy a new build so file names are updated. But this takes hours to arrange and perform, and all this time the site is partially broken.

Looks like the issue is reproduced for only a short time after deployment: we have never managed to obtain the broken JS or CSS from a Publisher manually, but still, it was cached on Dispatchers, so some visitor managed to get it. What can be the cause of this behavior?

Regards,

Alexander

[Fanindra_Surat]

Hi @Alexander_Prikhodko - Are you observing this issue on all the layers like AEM author, publisher, dispatcher and a CDN ?

[Alexander_Prikhodko]

Hi @Fanindra_Surat 

So far we have observed it on Dispatchers and CDN.

We haven't yet managed to reproduce it on Publisher on intent, but apparently it does happen there: since Dispatcher has JS with a new name cached, it means it requested it from Publisher after deployment, when Publisher had already generated the new name.

[Asutosh_Jena_ · Accepted solution]

Hi @Alexander_Prikhodko 

If I understood the issue, it seems like when you are trying to deploy a package with CSS and JS, you are able to reproduce the issue whereas if you are deploying evrything via a build it's not reproducible?

If my understanding is correct above, then it will be an issue with the dispatcher caching. When you install the package with CSS and JS updates, ensure to clear the dispatcher cache and it will pull the fresh content from publish instance with the new file name (hashed value).

Thanks!

[Alexander_Prikhodko]

Hi @Asutosh_Jena_ 

We deploy by installing a package that contains basically all parts of the app: components with their markup, styles and scripts, OSGi bundles, OSGi component configurations.

We clear Dispatcher cache each time after this; the issue is (as far as I can see), when a visitor goes to that Dispatcher shortly after deployment, the request to JS with a new name goes through to Publisher, and Publisher responds with JS having old contents, and this wrong state is cached on Dispatcher. When we discover this and try requesting Publisher for the same JS again, it responds with the correct version, but the wrong one is already cached on Dispatchers and event in visitors' browsers.

[Asutosh_Jena_]

Hi @Alexander_Prikhodko 

When you use ACS Commons Versioned clientlib, it will generate a random number so everytime after deployment the new number will be generated which will have the updated content.

We just need to ensure with each deployment, we are clearing both CDN and Dispatcher cache which will always fetch the latest content from publish and serve the latest content. Even if the content is cached at users browser, it will always use the new file as the new file will have a different hash.

Thanks!

[Ritesh_Mittal]

Hi @Alexander_Prikhodko ,

Are you facing this issue even after deployment, with new hash appended in JS URLs and still you see the old script loaded, that is quite odd since changing hash means it is not coming from cache and should give you latest code. Are you facing this issue on publish (non dispatcher URL) too. Also, are you setting any long TTL on your apache server. Does your deployment process covers dispatcher cache flush as well?

[Alexander_Prikhodko]

Hi @Ritesh_Mittal 

Yes, each time we deploy to Publisher, we also clear Dispatcher cache.

We haven't yet managed to reproduce it on Publisher on intent, but apparently it does happen sometimes: since Dispatcher has JS with a new name cached, it means it requested it from Publisher after deployment, when Publisher had already generated the new name.

[kchaurasiya]

@Alexander_Prikhodko Hi Alexander , As you are getting this issue on the CDN as well and you have already cleared the cache for the dispatcher. Did you try to clear the CDN cache.? if not then you can clear the cdn cache and verify.

Thank You.

[Alexander_Prikhodko]

Hi @kchaurasiya 

When the issue was reproduced, we also tried clearing CDN cache. It did help for new visitors, but the ones who visited the site during the issue had the wrong file cached in their browsers already. So I would like to find the root cause of this and prevent it in future.