Robots.txt file isn't accessible on publish environment AEM as a cloud service | Adobe Higher Education

JeffreyVSpringbok

Level 2

Robots.txt file isn't accessible on publish environment AEM as a cloud service

Forum|Forum|2 years ago
March 22, 2023
1 commentaire
2003 vue

Hi

We are implementing a robots.txt file that needs to be fetched from the dam (for several sites with several languages).

The first issue that we are facing is that we can't access the txt file on the publish environment with the direct path e.g. /content/dam/robots/<sitename>/en_gb/robots.txt.

When we try to put it on a link on a button same result (href attribute gets removed).
The same approach works fine with pdf files.

Does someone knows if we missed something regarding the filetypes that can't be published/accessed on the publish environment?

We are using AEM as a cloud service version : 2022.11.9850.20221116T162329Z.

Thanks

Ce sujet a été fermé aux réponses.

Community Advisor

Community Advisor

1. you need to write, redirect rule

https://github.com/arunpatidar02/aemaacs-aemlab/blob/5d778dd40cbcc899931f0570fb9b5e2a40a2d616/dispatcher.cloud/src/conf.d/rewrites/default_rewrite.rules#L12

2. allow robots.txt from dispatcher filter

Arun Patidar

JeffreyVSpringbokAuteur

Level 2

Hi Arun

We have enabled the right rules in the dispatcher but we still have the same issue.
As well if I bypass the dispatcher by accessing the publish url:

https://publish-pXXXXX-eXXXXX.adobeaemcloud.com/content/dam/robots/<sitename>/en_gb/robots.txt

It still gives a 404

If I try to access a jpg file it works.

Community Advisor

Community Advisor

Please check if this helps

https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410027

Arun Patidar