{"id":23830,"date":"2006-09-20T00:00:00","date_gmt":"2006-09-20T00:00:00","guid":{"rendered":"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-to-verify-googlebot\/"},"modified":"2006-09-20T00:00:00","modified_gmt":"2006-09-20T00:00:00","slug":"how-to-verify-googlebot","status":"publish","type":"ar_kb","link":"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-to-verify-googlebot\/","title":{"rendered":"How to verify Googlebot"},"content":{"rendered":"<aside class=\"key-point\">It&#8217;s been a while since we published this blog post. Some of the information may be outdated (for example, some images may be missing, and some links may not work anymore). Learn how you can <a href=\"https:\/\/alienroad.com\/google-bilgi-bankasi\/verify-requests-from-google-crawlers-and-fetchers\/\">Verify Googlebot<\/a>.<\/aside>\n<p class=\"gargardate\">Wednesday, September 20, 2006<\/p>\n<p>\n  Lately I&#8217;ve heard a couple smart<br \/>\n  <a href=\"https:\/\/www.crazyguyonabike.com\/\" class=\"external-link\">people<\/a> ask that<br \/>\n  search engines provide a way know that a bot is authentic. After all, any spammer could name<br \/>\n  their bot &#8220;Googlebot&#8221; and claim to be Google, so which bots do you trust and which do you block?\n<\/p>\n<p>\n  The common request we hear is to post a list of Googlebot IP addresses in some public place. The<br \/>\n  problem with that is that if\/when the IP ranges of our crawlers change, not everyone will know<br \/>\n  to check. In fact, the crawl team migrated Googlebot IPs a couple years ago and it was a real<br \/>\n  hassle alerting webmasters who had hard-coded an IP range. So the crawl folks have provided<br \/>\n  another way to authenticate Googlebot. Here&#8217;s an answer from one of the crawl people (quoted<br \/>\n  with their permission):\n<\/p>\n<blockquote>\n<p>\n    Telling webmasters to use DNS to verify on a case-by-case basis seems like the best way to go. I<br \/>\n    think the recommended technique would be to do a reverse DNS lookup, verify that the name is in<br \/>\n    the googlebot.com domain, and then do a corresponding forward DNS-&gt;IP lookup using that<br \/>\n    googlebot.com name. For example:\n  <\/p>\n<div><\/div>\n<p>\n    I don&#8217;t think just doing a reverse DNS lookup is sufficient, because a spoofer could set up<br \/>\n    reverse DNS to point to <code>crawl-a-b-c-d.googlebot.com<\/code>.\n  <\/p>\n<\/blockquote>\n<p>\n  This answer has also been provided to our help-desk, so I&#8217;d consider it an official way to<br \/>\n  authenticate Googlebot. In order to fetch from the &#8220;official&#8221; Googlebot IP range, the bot has to<br \/>\n  respect robots.txt and our internal hostload conventions so that Google doesn&#8217;t crawl you too<br \/>\n  hard.\n<\/p>\n<p>\n  (Thanks to N. and J. for help on this answer from the crawl side of things.)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It&#8217;s been a while since we published this blog post. Some of the information may be outdated (for example, some images may be missing, and some links may not work anymore). Learn how you can Verify Googlebot. Wednesday, September 20, 2006 Lately I&#8217;ve heard a couple smart people ask that search engines provide a way [&hellip;]<\/p>\n","protected":false},"menu_order":86589,"template":"","meta":{"footnotes":""},"ar_kb_kategori":[665],"ar_kb_etiket":[],"class_list":["post-23830","ar_kb","type-ar_kb","status-publish","has-post-thumbnail","hentry","ar_kb_kategori-blog"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/23830","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_kb"}],"version-history":[{"count":0,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/23830\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media\/26462"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=23830"}],"wp:term":[{"taxonomy":"ar_kb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_kategori?post=23830"},{"taxonomy":"ar_kb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_etiket?post=23830"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}