Robots.txt file isn't accessible on publish environment AEM as a cloud service | Community
Skip to main content
JeffreyVSpringbok
JeffreyVSpringbok
Level 2
March 22, 2023

Robots.txt file isn't accessible on publish environment AEM as a cloud service

  • March 22, 2023
  • 1 reply
  • 2003 views

Hi

 

We are implementing a robots.txt file that needs to be fetched from the dam (for several sites with several languages).

The first issue that we are facing is that we can't access the txt file on the publish environment with the direct path e.g. /content/dam/robots/<sitename>/en_gb/robots.txt.

 

When we try to put it on a link on a button same result (href attribute gets removed).
The same approach works fine with pdf files.

 

Does someone knows if we missed something regarding the filetypes that can't be published/accessed on the publish environment?

We are using AEM as a cloud service version : 2022.11.9850.20221116T162329Z.

 

 

Thanks

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

1 reply

arunpatidar
Community Advisor
arunpatidarCommunity Advisor
Community Advisor
March 22, 2023

1. you need to write, redirect rule

https://github.com/arunpatidar02/aemaacs-aemlab/blob/5d778dd40cbcc899931f0570fb9b5e2a40a2d616/dispatcher.cloud/src/conf.d/rewrites/default_rewrite.rules#L12

2. allow robots.txt from dispatcher filter

 

Arun Patidar
    JeffreyVSpringbok
    JeffreyVSpringbokAuthor
    Level 2
    April 14, 2023

    The issue was resolved by using the following filter:

     

    /0107 { /type "allow" /extension '(txt)' /path "/content/*" }

     

    and the following redirection in rewrite.rules:

     

    # Rewrite to robots.txt RewriteCond %{REQUEST_URI} /robots\.txt$ RewriteRule ^/(.*)$ /content/dam/robots/mysite/en_gb/robots.txt [PT,NC]​

     

      Terms & ConditionsAccessibility statement

      Loading footer...