Content AI - Crawler URL Parameter Whitelist | Community
Skip to main content
Level 2
May 20, 2019
New

Content AI - Crawler URL Parameter Whitelist

  • May 20, 2019
  • 1 reply
  • 1330 views

Ability to enable/white-list critical URL parameters for Content AI crawler to include. 

The current crawler ignores URL params to address duplicate content... OK understood. The ability to augment this with defined URL params would be incredibly helpful.

This is limiting even further, you can't add the content manually either since it's perceived as the same root URL... FacePalm! 

Currently we have a large collection of content that cannot be perceived by the Content AI crawler because of this limitation. I am sure there are other companies that would benefit from this enhancement.

1 reply

February 4, 2020
No text available