How to allow a robots.txt file to be visible on browser when accessed by url | Community
Skip to main content
R
Ralph8
Level 2
November 10, 2021
Solved

How to allow a robots.txt file to be visible on browser when accessed by url

  • November 10, 2021
  • 1 reply
  • 5276 views

How to allow a robots.txt file to be visible on browser when accessed by url
We are trying this on AEM as a cloud and in dispatcher we have already pasted the following config in dispatcher_vhost.conf but it still doesnt seem to work.
<Directory "/mnt/var/www/html/content/dam/sitefolder/">

<FilesMatch "\.(txt)$">

Header set Content-Disposition "inline"

Header set Cache-Control "max-age=86400, public"

Require all granted

</FilesMatch>

</Directory>

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Vijayalakshmi_S

Hi @ralph8,

By default, all the resources which has jcr:data property or jcr:content/jcr:data (which your robots.txt file will have in this case) will have an header with value as "attachment".

We need to make an entry (Exclude entry to not have the Content-disposition as attachment) in Apache Sling Content Disposition filter (in your publish instance) -> Clear cache, if desired and then try accessing the file. 

1 reply

Raja-Karuppsamy
Community Advisor
Raja-KaruppsamyCommunity Advisor
Community Advisor
November 10, 2021

@ralph8 

1. Please add the below rule in the /filter section of the farm file:

##Add allow rule for robots.txt in dispatcher
/0100 { /type "allow"  /url "/robots.txt"}

 

2. Add the redirect rule in the vhost file or rewrite file:

RewriteRule ^/robots.txt$  https://<dns>/<content-path>/robots.txt   (your content path should be masked based on content masking rule or mapping rule in publisher)

 

Now - try to access the robots.txt with public domain.

 

 

    R
    Ralph8Author
    Level 2
    November 10, 2021

    @raja-karuppsamy 
    robots.txt file is getting downloaded its still the same.
    I would need the file contents to be visible on browser on accessing..

      Raja-Karuppsamy
      Community Advisor
      Raja-KaruppsamyCommunity Advisor
      Community Advisor
      November 11, 2021

      It means you are able to access the robots.txt with public domain but the file is getting downloaded - right?

        Terms & ConditionsAccessibility statement

        Loading footer...