How to add robots.txt file when i do not have access to production CRDXLite?

[nandan_kumar]

Hello

How to add robots.txt file on when i do not have access to production CRDXLite?

Is there any way to add the file through configuration?

[smacdonald2008]

You need access to CRXDE lite to do this. See: http://www.wemblog.com/2013/06/how-to-implement-robotstxt-sitemapxml.html

[kautuk_sahni]

Hi 

As mentioned by Scott, this[1] is good article to start with.

Link:- http://www.wemblog.com/2013/06/how-to-implement-robotstxt-sitemapxml.html?spref=bl

// Go to CRXDE or CRXDE light, Or you can directly put them in your CVS under jcr_root folder. You can create different version of robots.txt based environment and domain name.

Then you can configure sling rewriter (org.apache.sling.jcr.resource.internal.JcrResourceResolverFactoryImpl.xml) under /apps/sling/config to redirect to correct robots.txt or any site specific configuration. More information about configuration can be obtained from here http://www.wemblog.com/2012/10/how-to-work-with-configurations-in-cq.html

For Prod something like resource.resolver.virtual="[/robots.txt:/robots-prod.txt] and for all other env
 resource.resolver.virtual="[/robots.txt:/robots-qa.txt]"

Extra reference link:- http://adobe-consulting-services.github.io/acs-aem-commons/features/simple-sitemap.html

I hope this would be helpful to you.

Thanks and Regards

Kautuk Sahni

[edubey]

Access to production publish crxde is disabled for security reasons but you might have access production author crxde where you can create and then replicate it.

Thanks

[tinyt6510512]

Hi 

I think you can use this way :

1. Create a field in page properties

ex : you can create a field Robots in page properties of HomePage template. ( the author can update & change it )

2. Under the homepage template.path :  \jcr_root\apps\<your-project>\components\pages\homepage, you can create a JSP file or HTML file ( robots.html or robots.jsp to get page properties ( example : ${pageProperties.robotsTxt}

3. You need to rewrite the url RewriteRule ^/robots.txt$  /content/test/home_pate.robots.html [PT]

Thanks,

Thuong.

[Peter_Puzanovs]

Why can't you just place robots.txt in your dam?

Then you would not need additional work apart from rewrite the url RewriteRule ^/robots.txt$  /content/dam/test/robots.txt [PT]

Regards,

Peter

[leeasling]

PuzanovsP wrote...

Why can't you just place robots.txt in your dam?

Then you would not need additional work apart from rewrite the url RewriteRule ^/robots.txt$  /content/dam/test/robots.txt [PT]

Regards,

Peter

I agree with this guy.  This makes it easier to update and you can restrict permissions to the file.

[Opkar_Gill · Accepted solution]

The robots.txt file is often a standard part of your site. I'm not sure why we are all recommending workarounds, the right way would be to work with your administrators and create the file.

Regards,

Opkar