Expand my Community achievements bar.

Don’t miss the AEM Skill Exchange in SF on Nov 14—hear from industry leaders, learn best practices, and enhance your AEM strategy with practical tips.

Mark Solution

This conversation has been locked.

prevent tracking servers html from appearing in search results

Avatar

Level 1

7/22/22

Description -

Please add the “noindex” robots meta tag to the <head> of the HTML or update the default robots.txt of omniture servers to disallow crawlers from indexing the site content.  Currently when using first party cookies that are hosted on omniture servers, google and bing will index those servers and return blank results for the pages (since there is no useful content on the tracking servers).

Why is this feature important to you -

We do not want omniture servers that are cnamed to our domain to appear in search results.

How would you like the feature to work -

update robots.txt 

instead of:

User-agent: *
Disallow:

it should be
User-agent: *
Disallow:/

Or better solution is to modify the servers to add the “noindex” robots meta tag to the <head> of the HTML

 

Current Behavior -

robots.txt allows googlebot to index the servers and the blank pages:  ssl.sc.omtrdc.net 63.140.36.104

When these servers are cnamed to our domain then it looks like our company is providing blank pages in search results

wphillips_0-1658514315639.png