Expand my Community achievements bar.

SOLVED

Dispatcher image caching issue.

Avatar

Level 2

We are facing an issue with dispatcher caching of image renditions, so wanted to poll the community on this -

Suppose image in question is A.jpg at path /content/dam/myapp/A.jpg.

  1. If the image rendition is accessed first, dispatcher caches the rendition at - /content/dam/myapp/A.jpg/jcr:content/renditions/some-rendition.jpeg, Here A.jpg is a directory
  2. Now if a request is made for A.jpg
  3. Dispatcher checks if the path /content/dam/myapp/A.jpg, exists ?
  4. Dispatcher gets a false positive as A.jpg is a directory, not a file.
  5. Dispatcher tries to return A.jpg from cache, but fails as A.jpg is a folder.

Wanted to check if anybody has faced this issue and how was it solved.

1 Accepted Solution

Avatar

Correct answer by
Level 8

The best solution would be to not refer to the renditions directly. As a general rule the recommended best practice is to refer to DAM assets through a servlet rather than directly. This gives you better control of the URL and allows you better cache control in general. 

So for example consider how the image rendering servlet in parbase works. It's image URLs look like /content/geometrixx/en/company/_jcr_content/par/3_1219086936652.img.jpg/1289480092880.jpg. The base servlet always returns the original rendition but it would be easy enough to add and additional selector into the pattern that was based on the rendition name and then write a rendition picker that selected the correct rendition. 

Absent that you can use the out of the box thumbnail servlet that is available when directly referencing a DAM asset. It only works if your assets follow the pattering of being named based on their dimensions but as long as you are using the out of the box rendition creation functionality they should be compatible. The path to this approach looks like this: /content/dam/geometrixx/portraits/scott_reynolds.jpg.thumb.100.140.png. One key thing to keep in mind is that this servlet expects the image parameters to be in the reverse order of their rendition name. So with the example I just used the full path to the rendition is /content/dam/geometrixx/portraits/scott_reynolds.jpg/jcr:content/renditions/cq5dam.thumbnail.140.100.png. Notice how the order of the dimensions is reversed. /content/dam/geometrixx/portraits/scott_reynolds.jpg.thumb.140.100.png will return  error (actually a generic icon).

There is one thing to be aware of when using the out of the box thumbnail servlet - it opens yo up to DDOS attacks since you can vary the dimensions infinitely and keep caching the generic icon under different names. This both puts you publish server under additional load and potentially uses up all your web server disk space. You either need to white list the dimension selectors you want in your dispatcher filter section, or write you own servlet with descriptive names to match the dimensions instead of actual dimensions. 

Also another thing to be aware of  - it's possible to create assets without an extension in them - if you follow the normal procedures for uploading an image it doesn't happen, however it should be noted that it's possible. For example look at this assets that ships with geometrixx sites - /content/dam/projects/media/cover - it's an image but the path does not include an extension. If you try just adding an extension you get an error - so relying on your DAM assets to have extensions in their name is not guaranteed to work. 99% of the time you are OK but you should keep in mind it's possible to have and asset without an extension in it's name. Better to follow the pattern of using a servlet to render you images instead of direct references.  

View solution in original post

9 Replies

Avatar

Level 3

We have the same issue and struggling with it right now. Solution mentioned by bsloki is not exactly a solution but workaround, although it helps to prevent the issue if it is related to the specific component but doesn't resolve the core problem.

Avatar

Level 2

Thank you for your response!

bsloki, problem with the approach mentioned in these articles is that if original image is accessed with .../jcr:content/renditions/original. these images wont be cached in absence of extension.

Other solution that I have is, to create 2 different mappings(etc->map) for renditions and original 

  1. /content/dam/myapp/A.jpg to /images/A.jpg
  2.  /content/dam/myapp/A.jpg/jcr:content/renditions/some-rendition.jpeg to /renditionserver/A.jpg/jcr:content/renditions/some-rendition.jpeg

What this'll do is, create 2 different folder structure in the dispatcher cache, thus resolving the issue that we were facing.

Abhinav

Avatar

Correct answer by
Level 8

The best solution would be to not refer to the renditions directly. As a general rule the recommended best practice is to refer to DAM assets through a servlet rather than directly. This gives you better control of the URL and allows you better cache control in general. 

So for example consider how the image rendering servlet in parbase works. It's image URLs look like /content/geometrixx/en/company/_jcr_content/par/3_1219086936652.img.jpg/1289480092880.jpg. The base servlet always returns the original rendition but it would be easy enough to add and additional selector into the pattern that was based on the rendition name and then write a rendition picker that selected the correct rendition. 

Absent that you can use the out of the box thumbnail servlet that is available when directly referencing a DAM asset. It only works if your assets follow the pattering of being named based on their dimensions but as long as you are using the out of the box rendition creation functionality they should be compatible. The path to this approach looks like this: /content/dam/geometrixx/portraits/scott_reynolds.jpg.thumb.100.140.png. One key thing to keep in mind is that this servlet expects the image parameters to be in the reverse order of their rendition name. So with the example I just used the full path to the rendition is /content/dam/geometrixx/portraits/scott_reynolds.jpg/jcr:content/renditions/cq5dam.thumbnail.140.100.png. Notice how the order of the dimensions is reversed. /content/dam/geometrixx/portraits/scott_reynolds.jpg.thumb.140.100.png will return  error (actually a generic icon).

There is one thing to be aware of when using the out of the box thumbnail servlet - it opens yo up to DDOS attacks since you can vary the dimensions infinitely and keep caching the generic icon under different names. This both puts you publish server under additional load and potentially uses up all your web server disk space. You either need to white list the dimension selectors you want in your dispatcher filter section, or write you own servlet with descriptive names to match the dimensions instead of actual dimensions. 

Also another thing to be aware of  - it's possible to create assets without an extension in them - if you follow the normal procedures for uploading an image it doesn't happen, however it should be noted that it's possible. For example look at this assets that ships with geometrixx sites - /content/dam/projects/media/cover - it's an image but the path does not include an extension. If you try just adding an extension you get an error - so relying on your DAM assets to have extensions in their name is not guaranteed to work. 99% of the time you are OK but you should keep in mind it's possible to have and asset without an extension in it's name. Better to follow the pattern of using a servlet to render you images instead of direct references.  

Avatar

Level 2

Thanks Orotas !

Can you please comment on the approach that I had mentioned to solve the caching issue, using etc mapping to create to different hierarchies for original and renditions. Etc map solution appears to be a lot simpler to fix an already deployed solution.

If I had to redo the implementation, I might go with the approach you advised. So wanted to check your thoughts on the etc based solution.

Avatar

Level 3

Orotas, it all makes sense for images but what to do with other assets? We are facing described problem for .pdf files. System creates thumbnails for them and extracts metadata, but we have to serve original files, not renditions...

Avatar

Level 8

If you can get the regular expression right you could probably get the rewrites working. Personally i have always found complex rewrites in /etc/maps challenging to get working as expected. If you do get it working you should be aware that your cache flushing may not work as expected since the path in the cache won't match the path in repository. 

Avatar

Level 8

Other asset types could work pretty much the same way - you'd have to write your servlet and figure out how to map selectors to the page specific renditions. Unlike the images you would not find as many helper APIs since most people do reference their PDFs directly these days. Still the underlying concept isn't any different. If you follow /libs/foundation/components/parbase/img.GET.java as an example you remove all the image specific stuff, you add the code to map selectors to renditions (and code to return 404s to reduce the impact of a DDOS attack. Then you us com.day.cq.wcm.foundation.Download instead of com.day.cq.wcm.foundation.Image to get the binary file, and add code to get the mime type correct. If you have access to any old versions of CQ (back to maybe 5.2 - or 4.x) you could probably dig up some example code. 

Avatar

Level 2

Hi Abhinav,

What was the final solution you went with? Did you handle PDF assets as well?

I am facing the same dispatcher cache issue.

Thanks.