Google robots.txt Tester | Community
Skip to main content
Pete_Jones
Level 2
July 27, 2018
Solved

Google robots.txt Tester

  • July 27, 2018
  • 1 reply
  • 6922 views

We have several pages on our website that use Marketo forms. Google's robot.txt testers are not seeing the Marketo form data and have treated the blocked resource to impact indexing of the pages. They say all we need to do is verify ownership of the blocked resource, our Marketo forms. I'm curious if there are any best practices for uploading the HTML file to our Marketo instance and displaying it for Google to see. The domain where we display our files loaded to Marketo and the Marketo domain we use to link to the forms are two separate domains so Google cannot verify them.

Best answer by SanfordWhiteman

Sanford. Okay, we have the SSL cert added to the Marketo domain now but still seeing this issue. For example, the domain below. When Google indexes the page with their robots they do not show the form and actally, the form is Blocked in the Google Search Console. Thoughts?

Parachurch Organization Program | GuideOne Insurance


You haven't changed your form embed code. You need to change both references to //app-ab13.marketo.com to your now-SSL-protected Marketo LP domain.

1 reply

SanfordWhiteman
Level 10
July 27, 2018

I'm curious if there are any best practices for uploading the HTML file to our Marketo instance and displaying it for Google to see.

That question doesn't make sense.

You can manage your Marketo LP domain's robots.txt like so: https://blog.teknkl.com/customizing-the-robots-txt-for-marketo-lps/

Pete_Jones
Level 2
July 27, 2018

I am not interested in managing the Marketo LP domains. I am interested in providing verification of ownership of the Marketo forms I have embedded on my website. Google is not seeing the forms and as a result, it is blocking the robot.txt file from completing the crawl. One way to verify ownership in Google Search Console is to upload the HTML file to Marketo and display so Google can see it and complete the crawl. I am curious if that has been done and if there are best practices available.

SanfordWhiteman
Level 10
July 27, 2018

You create an LP with the verification HTML as the content.

There's no other practice for creating a page at the root of your Marketo LP domain.