<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Facing Issues for Robots.txt in AEM Cloud SDK in Adobe Experience Manager Discussions</title>
    <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410116#M35201</link>
    <description>&lt;P&gt;HI &lt;LI-USER uid="17439118"&gt;&lt;/LI-USER&gt;&amp;nbsp;Please apply the below rule in &lt;STRONG&gt;filter.any&lt;/STRONG&gt; file between rule 0008 and 0010.&lt;/P&gt;&lt;PRE&gt;/0009 { /glob "/content/*.txt" /type "allow" }&lt;/PRE&gt;&lt;P&gt;Restart the dispatcher and test it.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Thanks!&lt;/P&gt;</description>
    <pubDate>Thu, 27 May 2021 17:04:23 GMT</pubDate>
    <dc:creator>Asutosh_Jena_</dc:creator>
    <dc:date>2021-05-27T17:04:23Z</dc:date>
    <item>
      <title>Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410027#M35196</link>
      <description>&lt;P&gt;Dear All,&lt;/P&gt;&lt;P&gt;I have setup AEM cloud SDK in my local and trying to implement robots.txt by following the below blog.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;A href="https://www.aemtutorial.info/2020/07/" target="_blank"&gt;https://www.aemtutorial.info/2020/07/&lt;/A&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Here I am facing 2 issues.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;1) Issue-1 - I have created a file under root content like below.&lt;/P&gt;&lt;P&gt;/content/mysite/robots.txt&lt;/P&gt;&lt;P&gt;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;When I am trying to see the robots.txt from the page in author/publish like&amp;nbsp;&lt;A href="http://localhost:4503/content/mysite/robots.txt" target="_blank"&gt;http://localhost:4503/content/mysite/robots.txt&lt;/A&gt; , then robots.txt is downloading...&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;FONT color="#000000"&gt;2)&amp;nbsp;&lt;/FONT&gt;Issue-2 - When I am hitting the robots.txt from the dispatcher page then also I am not seeing the any content frrom robots.txt , as shown below.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_0-1622106004936.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31971i9DC0703E4AA0AD1B/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_0-1622106004936.png" alt="sunitaBorn_0-1622106004936.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I am getting below error log in dispatcher it is showing that&amp;nbsp;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;blocked [publishfarm/-] 0ms "localhost:8082".&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;[27/May/2021:08:39:31 +0000] "GET /content/dam/mysite/robots.txt HTTP/1.1" - blocked [publishfarm/-] 0ms "localhost:8082"&lt;/STRONG&gt;&lt;/FONT&gt;&lt;BR /&gt;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;172.17.0.1 "localhost:8082" - [27/May/2021:08:39:31 +0000] "GET /content/dam/mysite/robots.txt HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36"&lt;/STRONG&gt;&lt;/FONT&gt;&lt;BR /&gt;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;172.17.0.1 "localhost:8082" - [27/May/2021:08:40:25 +0000] "GET /content/mysite/robots.txt HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36"&lt;/STRONG&gt;&lt;/FONT&gt;&lt;BR /&gt;&lt;FONT color="#FF0000"&gt;&lt;STRONG&gt;[27/May/2021:08:40:25 +0000] "GET /content/mysite/robots.txt HTTP/1.1" - blocked [publishfarm/-] 0ms "localhost:8082"&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;FONT color="#000000"&gt;My robots.txt file is below&lt;/FONT&gt;&lt;/P&gt;&lt;P&gt;&lt;FONT color="#000000"&gt;#Any search crawler can c&lt;/FONT&gt;rawl our site&lt;BR /&gt;User-agent: *&lt;/P&gt;&lt;P&gt;#Allow only below mentioned paths&lt;BR /&gt;Allow: /en/&lt;BR /&gt;Allow: /fr/&lt;BR /&gt;Allow: /gb/&lt;BR /&gt;Allow: /in/&lt;BR /&gt;#Disallow everything else&lt;BR /&gt;Disallow: /&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Can anybody please help me on this. Thanks a lot....NOTE that I am using AEM cloud SDK in my local.&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 09:04:39 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410027#M35196</guid>
      <dc:creator>sunitaBorn</dc:creator>
      <dc:date>2021-05-27T09:04:39Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410035#M35197</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;LI-USER uid="17439118"&gt;&lt;/LI-USER&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Please see my answers below:&lt;/P&gt;&lt;P&gt;#1: You need to configure the ContentDispositionFilter with the file path as exclusion to open the file instead of downloading it. Please add the below config in config.publish run mode so on all publish instance it will be opened while in author it will be downloaded. If you want in author also to open you can add the same configuration to config it self. But it's only required on publish as publish instances is exposed to public.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;org.apache.sling.security.impl.ContentDispositionFilter.xml&lt;/STRONG&gt;&lt;/P&gt;&lt;PRE&gt;&lt;SPAN&gt;&amp;lt;?&lt;/SPAN&gt;&lt;SPAN&gt;xml version&lt;/SPAN&gt;&lt;SPAN&gt;="1.0" &lt;/SPAN&gt;&lt;SPAN&gt;encoding&lt;/SPAN&gt;&lt;SPAN&gt;="UTF-8"&lt;/SPAN&gt;&lt;SPAN&gt;?&amp;gt;&lt;BR /&gt;&lt;/SPAN&gt;&lt;SPAN&gt;&amp;lt;&lt;/SPAN&gt;&lt;SPAN&gt;jcr&lt;/SPAN&gt;&lt;SPAN&gt;:root &lt;/SPAN&gt;&lt;SPAN&gt;xmlns:&lt;/SPAN&gt;&lt;SPAN&gt;sling&lt;/SPAN&gt;&lt;SPAN&gt;="http://sling.apache.org/jcr/sling/1.0" &lt;/SPAN&gt;&lt;SPAN&gt;xmlns:&lt;/SPAN&gt;&lt;SPAN&gt;jcr&lt;/SPAN&gt;&lt;SPAN&gt;="http://www.jcp.org/jcr/1.0"&lt;BR /&gt;&lt;/SPAN&gt;    &lt;SPAN&gt;jcr&lt;/SPAN&gt;&lt;SPAN&gt;:primaryType&lt;/SPAN&gt;&lt;SPAN&gt;="sling:OsgiConfig"&lt;BR /&gt;&lt;/SPAN&gt;    &lt;SPAN&gt;sling.content.disposition.all.paths&lt;/SPAN&gt;&lt;SPAN&gt;="{Boolean}false"&lt;BR /&gt;&lt;/SPAN&gt;    &lt;SPAN&gt;sling.content.disposition.excluded.paths&lt;/SPAN&gt;&lt;SPAN&gt;="[&lt;STRONG&gt;/content/mysite/robots.txt&lt;/STRONG&gt;]"&lt;/SPAN&gt;&lt;SPAN&gt;/&amp;gt;&lt;/SPAN&gt;&lt;/PRE&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;#2: I see the path is blocked in "&lt;STRONG&gt;publishfarm&lt;/STRONG&gt;" farm file. You need to enable access to the file location i.e. /content/mysite/* to allow the file to load. Ideally there will be a redirect set at the dispatcher because the file will be always accessed like &lt;A href="http://www.website.com/robots.txt" target="_blank" rel="noopener"&gt;www.website.com/robots.txt &lt;/A&gt;and it should serve content from the actual location. So you need to apply the below redirect at the dispatcher as well:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;PRE&gt;RewriteCond %{REQUEST_URI} ^/robots.txt$&lt;BR /&gt;RewriteRule (.*) /content/mysite/robots.txt  [PT,L]&lt;/PRE&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Thanks!&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 10:03:25 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410035#M35197</guid>
      <dc:creator>Asutosh_Jena_</dc:creator>
      <dc:date>2021-05-27T10:03:25Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410058#M35198</link>
      <description>&lt;P&gt;Thanks &lt;LI-USER uid="7532759"&gt;&lt;/LI-USER&gt;..Now my issue 1 is fixed and struggling for issue 2.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;My cloud setup is like below.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I have conf.d and conf.dispatcher.d folder as shown in below.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_0-1622121393124.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31976i67BAF154DFA5B3FA/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_0-1622121393124.png" alt="sunitaBorn_0-1622121393124.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_1-1622121447635.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31977iF936FDA0626B9B27/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_1-1622121447635.png" alt="sunitaBorn_1-1622121447635.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;I have done the below steps and still seeing the below error.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Step-1 , I have created a mysite.farm like below.&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_2-1622121651517.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31978iB628F1152339D092/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_2-1622121651517.png" alt="sunitaBorn_2-1622121651517.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Step-2 Then include&amp;nbsp;and include this farm inside enabled_farms like below.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;../available_farms/default.farm&lt;BR /&gt;../available_farms/mysite.farm&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_3-1622121824366.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31979i7E1A8DCD805E41A8/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_3-1622121824366.png" alt="sunitaBorn_3-1622121824366.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Step-3 Create a mysite.vhost inside avialble_vhosts , as shown below.&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_4-1622122009845.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31980iE89CC93655DC1D9C/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_4-1622122009845.png" alt="sunitaBorn_4-1622122009845.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Step-4 I have added the above 2 lines inside mysite.vhost file, as shown below in bold text.&lt;/P&gt;&lt;P&gt;#&lt;BR /&gt;# This is the default publish virtualhost definition for Apache.&lt;BR /&gt;#&lt;BR /&gt;# DO NOT EDIT this file, your changes will have no impact on your deployment.&lt;BR /&gt;#&lt;BR /&gt;# Instead create a copy in the folder conf.d/available_vhosts and edit the copy.&lt;BR /&gt;# Finally, change to the directory conf.d/enabled_vhosts, remove the symbolic&lt;BR /&gt;# link for default.vhost and create a symbolic link to your copy.&lt;BR /&gt;#&lt;/P&gt;&lt;P&gt;# Include customer defined variables&lt;BR /&gt;Include conf.d/variables/custom.vars&lt;/P&gt;&lt;P&gt;&amp;lt;VirtualHost *:80&amp;gt;&lt;BR /&gt;ServerName "publish"&lt;BR /&gt;# Put names of which domains are used for your published site/content here&lt;BR /&gt;ServerAlias "*"&lt;BR /&gt;# Use a document root that matches the one in conf.dispatcher.d/default.farm&lt;BR /&gt;DocumentRoot "${DOCROOT}"&lt;BR /&gt;# URI dereferencing algorithm is applied at Sling's level, do not decode parameters here&lt;BR /&gt;AllowEncodedSlashes NoDecode&lt;BR /&gt;# Add header breadcrumbs for help in troubleshooting&lt;BR /&gt;&amp;lt;IfModule mod_headers.c&amp;gt;&lt;BR /&gt;Header add X-Vhost "publish"&lt;BR /&gt;&amp;lt;/IfModule&amp;gt;&lt;BR /&gt;&amp;lt;Directory /&amp;gt;&lt;BR /&gt;&amp;lt;IfModule disp_apache2.c&amp;gt;&lt;BR /&gt;# Some items cache with the wrong mime type&lt;BR /&gt;# Use this option to use the name to auto-detect mime types when cached improperly&lt;BR /&gt;ModMimeUsePathInfo On&lt;BR /&gt;# Use this option to avoid cache poisioning&lt;BR /&gt;# Sling will return /content/image.jpg as well as /content/image.jpg/ but apache can't search /content/image.jpg/ as a file&lt;BR /&gt;# Apache will treat that like a directory. This assures the last slash is never stored in cache&lt;BR /&gt;DirectorySlash Off&lt;BR /&gt;# Enable the dispatcher file handler for apache to fetch files from AEM&lt;BR /&gt;SetHandler dispatcher-handler&lt;BR /&gt;&amp;lt;/IfModule&amp;gt;&lt;BR /&gt;Options FollowSymLinks&lt;BR /&gt;AllowOverride None&lt;BR /&gt;# Insert filter&lt;BR /&gt;SetOutputFilter DEFLATE&lt;BR /&gt;# Don't compress images&lt;BR /&gt;SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary&lt;BR /&gt;# Prevent clickjacking&lt;BR /&gt;Header always append X-Frame-Options SAMEORIGIN&lt;BR /&gt;&amp;lt;/Directory&amp;gt;&lt;BR /&gt;&amp;lt;Directory "${DOCROOT}"&amp;gt;&lt;BR /&gt;AllowOverride None&lt;BR /&gt;Require all granted&lt;BR /&gt;&amp;lt;/Directory&amp;gt;&lt;BR /&gt;&amp;lt;IfModule disp_apache2.c&amp;gt;&lt;BR /&gt;# Enabled to allow rewrites to take affect and not be ignored by the dispatcher module&lt;BR /&gt;DispatcherUseProcessedURL On&lt;BR /&gt;# Default setting to allow all errors to come from the aem instance&lt;BR /&gt;DispatcherPassError 0&lt;BR /&gt;&amp;lt;/IfModule&amp;gt;&lt;BR /&gt;&amp;lt;IfModule mod_rewrite.c&amp;gt;&lt;BR /&gt;RewriteEngine on&lt;BR /&gt;Include conf.d/rewrites/rewrite.rules&lt;/P&gt;&lt;P&gt;# Rewrite index page internally, pass through (PT)&lt;BR /&gt;RewriteRule "^(/?)$" "/index.html" [PT]&lt;BR /&gt;&lt;BR /&gt;&lt;STRONG&gt;RewriteCond %{REQUEST_URI} ^/robots.txt$&lt;/STRONG&gt;&lt;BR /&gt;&lt;STRONG&gt;RewriteRule (.*) /content/myraitt/us/en/robots.txt [PT,L]&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&amp;lt;/IfModule&amp;gt;&lt;BR /&gt;&amp;lt;/VirtualHost&amp;gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Step-5 Also I have changed the below in mysite_rewrite_rules, as shown below&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;#&lt;/P&gt;&lt;P&gt;# Examples:&lt;BR /&gt;# This ruleset would look for robots.txt and fetch it from the dam only if the domain is exampleco-dev.adobecqms.net&lt;BR /&gt;&lt;BR /&gt;&lt;STRONG&gt;RewriteCond %{SERVER_NAME} localhost:8080 [NC] &lt;/STRONG&gt;&lt;BR /&gt;&lt;STRONG&gt;RewriteRule ^/robots.txt$ /content/dam/myraitt/robots.txt [NC,PT]&lt;/STRONG&gt;&lt;BR /&gt;&lt;BR /&gt;# This ruleset would look for favicon.ico in exampleco's base dam folder if the domain is exampleco-brand1-dev.adobecqms.net&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_5-1622122218893.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31981i2769E678076F024A/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_5-1622122218893.png" alt="sunitaBorn_5-1622122218893.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Still I am getting below error blocked by publishfarm , as shown below...Anything I am missing here.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;BR /&gt;&lt;STRONG&gt;172.17.0.1 "localhost:8080" - [27/May/2021:13:12:36 +0000] "GET /content/dam/myraitt/robots.txt HTTP/1.1" 404 196 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36"&lt;/STRONG&gt;&lt;BR /&gt;&lt;STRONG&gt;[27/May/2021:13:12:36 +0000] "GET /content/dam/myraitt/robots.txt HTTP/1.1" - blocked [publishfarm/-] 0ms "localhost:8080"&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 13:33:05 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410058#M35198</guid>
      <dc:creator>sunitaBorn</dc:creator>
      <dc:date>2021-05-27T13:33:05Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410085#M35199</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;LI-USER uid="17439118"&gt;&lt;/LI-USER&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Can you please share the file content from filters folder? The path is blocked which is why you are not able to access it.&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 14:36:35 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410085#M35199</guid>
      <dc:creator>Asutosh_Jena_</dc:creator>
      <dc:date>2021-05-27T14:36:35Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410088#M35200</link>
      <description>&lt;P&gt;Hi &lt;LI-USER uid="7532759"&gt;&lt;/LI-USER&gt;..Please see my file from filters folder.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_1-1622127552933.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/31989i06C3741859FC59D1/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_1-1622127552933.png" alt="sunitaBorn_1-1622127552933.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;*********************default-filters.any*************************&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;#&lt;BR /&gt;# This is the default filter ACL specifying what requests are handled by the dispatcher.&lt;BR /&gt;#&lt;BR /&gt;# DO NOT EDIT this file, your changes will have no impact on your deployment.&lt;BR /&gt;#&lt;BR /&gt;# Instead modify filters.any.&lt;BR /&gt;#&lt;/P&gt;&lt;P&gt;# deny everything and allow specific entries&lt;BR /&gt;# Start with everything blocked as a safeguard and open things customers need and what's safe OOTB&lt;BR /&gt;/0001 { /type "deny" /url "*" }&lt;/P&gt;&lt;P&gt;# Open consoles if this isn't a production environment by uncommenting the next few lines&lt;BR /&gt;# /002 { /type "allow" /url "/crx/*" } # allow content repository&lt;BR /&gt;# /003 { /type "allow" /url "/system/*" } # allow OSGi console&lt;/P&gt;&lt;P&gt;# allow non-public content directories if this isn't a production environment by uncommenting the next few lines&lt;BR /&gt;# /004 { /type "allow" /url "/apps/*" } # allow apps access&lt;BR /&gt;# /005 { /type "allow" /url "/bin/*" } # allow bin path access&lt;/P&gt;&lt;P&gt;# This rule allows content to be access&lt;BR /&gt;/0010 { /type "allow" /extension '(css|eot|gif|ico|jpeg|jpg|js|gif|pdf|png|svg|swf|ttf|woff|woff2|html)' /path "/content/*" }&lt;BR /&gt;# disable this rule to allow mapped content only&lt;/P&gt;&lt;P&gt;# Enable specific mime types in non-public content directories&lt;BR /&gt;/0011 { /type "allow" /method "GET" /extension '(css|eot|gif|ico|jpeg|jpg|js|gif|png|svg|swf|ttf|woff|woff2)' }&lt;/P&gt;&lt;P&gt;# Enable clientlibs proxy servlet&lt;BR /&gt;/0012 { /type "allow" /method "GET" /url "/etc.clientlibs/*" }&lt;/P&gt;&lt;P&gt;# Enable basic features&lt;BR /&gt;/0013 { /type "allow" /method "GET" /url '/libs/granite/csrf/token.json' /extension 'json' } # AEM provides a framework aimed at preventing Cross-Site Request Forgery attacks&lt;BR /&gt;/0014 { /type "allow" /method "POST" /url "/content/*.form.html" } # allow POSTs to form selectors under content&lt;/P&gt;&lt;P&gt;/0015 { /type "allow" /method "GET" /path "/libs/cq/personalization" } # enable personalization&lt;BR /&gt;/0016 { /type "allow" /method "POST" /path "/content/*.commerce.cart.json" } # allow POSTs to update the shopping cart&lt;/P&gt;&lt;P&gt;# Deny content grabbing for greedy queries and prevent un-intended self DOS attacks&lt;BR /&gt;/0017 { /type "deny" /selectors '(feed|rss|pages|languages|blueprint|infinity|tidy|sysview|docview|query|[0-9-]+|jcr:content)' /extension '(json|xml|html|feed)' }&lt;/P&gt;&lt;P&gt;# Deny authoring query params&lt;BR /&gt;/0018 { /type "deny" /method "GET" /query "debug=*" }&lt;BR /&gt;/0019 { /type "deny" /method "GET" /query "wcmmode=*" }&lt;/P&gt;&lt;P&gt;# Allow current user&lt;BR /&gt;/0020 { /type "allow" /url "/libs/granite/security/currentuser.json" }&lt;/P&gt;&lt;P&gt;# Allow index page&lt;BR /&gt;/0030 { /type "allow" /url "/index.html" }&lt;/P&gt;&lt;P&gt;# Allow IMS Authentication&lt;BR /&gt;/0031 { /type "allow" /method "GET" /url "/callback/j_security_check" }&lt;/P&gt;&lt;P&gt;# AEM Forms specific filters&lt;BR /&gt;# to allow AF specific endpoints for prefill, submit and sign&lt;BR /&gt;/0032 { /type "allow" /path "/content/forms/af/*" /method "POST" /selectors '(submit|internalsubmit|agreement|signSubmit|prefilldata)' /extension '(jsp|json)' }&lt;/P&gt;&lt;P&gt;# to allow AF specific endpoints for thank you page&lt;BR /&gt;/0033 { /type "allow" /path "/content/forms/af/*" /method "GET" /selectors '(guideThankYouPage|guideAsyncThankYouPage)' /extension '(html)'}&lt;/P&gt;&lt;P&gt;# to allow AF specific endpoints for lazy loading&lt;BR /&gt;/0034 { /type "allow" /path "/content/forms/af/*" /method "GET" /extension '(jsonhtmlemitter)'}&lt;/P&gt;&lt;P&gt;# to allow fp related functionalities&lt;BR /&gt;/0035 { /type "allow" /path "/content/forms/*" /selectors '(fp|attach|draft|dor|api)' /extension '(html|jsp|json|pdf)' }&lt;/P&gt;&lt;P&gt;# to allow forms access via dam path&lt;BR /&gt;/0036 { /type "allow" /path "/content/dam/formsanddocuments/**/jcr:content" /method "GET"}&lt;/P&gt;&lt;P&gt;# to allow invoke service functionality (FDM)&lt;BR /&gt;/0037 { /type "allow" /path "/content/forms/*" /selectors '(af)' /extension '(dermis)' }&lt;/P&gt;&lt;P&gt;# AEM Screens Filters&lt;BR /&gt;# to allow AEM Screens channels selectors&lt;BR /&gt;/0050 { /type "allow" /method "GET" /url "/screens/channels.json" }&lt;/P&gt;&lt;P&gt;# to allow AEM Screens Content and selectors&lt;BR /&gt;/0051 { /type "allow" /method '(GET|HEAD)' /url "/content/screens/*" }&lt;/P&gt;&lt;P&gt;# AEM Sites Filters&lt;BR /&gt;# to allow site30 theme servlet&lt;BR /&gt;/0052 { /type "allow" /extension "theme" /path "/content/*" }&lt;/P&gt;&lt;P&gt;# Allow GraphQL &amp;amp; preflight requests&lt;BR /&gt;# GraphQL also supports "GET" requests, if you intend to use "GET" add a rule in filters.any&lt;BR /&gt;/0060 { /type "allow" /method '(POST|OPTIONS)' /url "/content/_cq_graphql/*/endpoint.json" }&lt;/P&gt;&lt;P&gt;# GraphQL Persisted Queries &amp;amp; preflight requests&lt;BR /&gt;/0061 { /type "allow" /method '(GET|POST|OPTIONS)' /url "/graphql/execute.json*" }&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;*******************filters.any*******************&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;#&lt;BR /&gt;# This file contains the filter ACL, and can be customized.&lt;BR /&gt;#&lt;BR /&gt;# By default, it includes the default filter ACL.&lt;BR /&gt;#&lt;/P&gt;&lt;P&gt;$include "./default_filters.any"&lt;/P&gt;&lt;P&gt;#/006 { /type "allow" /url "/content/myraitt/us/en/robots.txt"} # allow robots.txt path access&lt;BR /&gt;#/007 { /type "allow" /url "/robots.txt"}&lt;BR /&gt;#/008 { /type "allow" /url "/content/dam/myraitt/robots.txt"}&lt;/P&gt;&lt;P&gt;/0010 { /type "allow" /extension '(css|eot|gif|ico|jpeg|jpg|js|gif|pdf|png|svg|swf|ttf|woff|woff2|html|txt)' /path "/content/*" }&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 15:00:32 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410088#M35200</guid>
      <dc:creator>sunitaBorn</dc:creator>
      <dc:date>2021-05-27T15:00:32Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410116#M35201</link>
      <description>&lt;P&gt;HI &lt;LI-USER uid="17439118"&gt;&lt;/LI-USER&gt;&amp;nbsp;Please apply the below rule in &lt;STRONG&gt;filter.any&lt;/STRONG&gt; file between rule 0008 and 0010.&lt;/P&gt;&lt;PRE&gt;/0009 { /glob "/content/*.txt" /type "allow" }&lt;/PRE&gt;&lt;P&gt;Restart the dispatcher and test it.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;Thanks!&lt;/P&gt;</description>
      <pubDate>Thu, 27 May 2021 17:04:23 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410116#M35201</guid>
      <dc:creator>Asutosh_Jena_</dc:creator>
      <dc:date>2021-05-27T17:04:23Z</dc:date>
    </item>
    <item>
      <title>Re: Facing Issues for Robots.txt in AEM Cloud SDK</title>
      <link>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410586#M35202</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;LI-USER uid="7532759"&gt;&lt;/LI-USER&gt;,&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;When I used&amp;nbsp;&amp;nbsp;&lt;STRONG&gt;/0009 { /glob "/content/*.txt" /type "allow" }&lt;/STRONG&gt; in filter.any , then I am getting below error.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;FONT color="#FF0000"&gt;Cloud manager validator 2.0.30&lt;/FONT&gt;&lt;BR /&gt;&lt;FONT color="#FF0000"&gt;2021/06/01 16:20:38 Dispatcher configuration validation failed:&lt;/FONT&gt;&lt;BR /&gt;&lt;FONT color="#FF0000"&gt;conf.dispatcher.d\filters\filters.any:9: filter must not use glob pattern to allow requests&lt;/FONT&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_0-1622544802529.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/32063iCF2D51ABA53DAFB3/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_0-1622544802529.png" alt="sunitaBorn_0-1622544802529.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;When I commented&amp;nbsp;&amp;nbsp;&lt;STRONG&gt;/009 { /glob "/content/*.txt" /type "allow" } and used&amp;nbsp;/009 { /type "allow" /extension '(txt)' /path "/content/myraitt/*" } then I am able to build success and I am able to see my robots.txt with 200 , as shown below.&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_1-1622545415948.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/32064iEE5B6BC29D82576F/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_1-1622545415948.png" alt="sunitaBorn_1-1622545415948.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="sunitaBorn_2-1622545438769.png" style="width: 400px;"&gt;&lt;img src="https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/32065iE9695CC7010B656D/image-size/medium/is-moderation-mode/true?v=v2&amp;amp;px=400" role="button" title="sunitaBorn_2-1622545438769.png" alt="sunitaBorn_2-1622545438769.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Tue, 01 Jun 2021 11:05:06 GMT</pubDate>
      <guid>https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/facing-issues-for-robots-txt-in-aem-cloud-sdk/m-p/410586#M35202</guid>
      <dc:creator>sunitaBorn</dc:creator>
      <dc:date>2021-06-01T11:05:06Z</dc:date>
    </item>
  </channel>
</rss>

