Friday, November 24, 2023<\/p>\n
\n The crawl rate limiter tool in Search Console is being deprecated on Jan 8th, 2024. This tool has \n Googlebot reacts to how the site–or more specifically the server handling the site– \n In contrast, the rate limiter tool had a much slower effect; in fact it may have taken over a day \n Because of the advances in the automated crawl rate handling, and in the spirit of keeping things \n If you have questions or comments, write in our
\n been
\n available for over a decade<\/a>,
\n but with the improvements we’ve made to our crawling logic and other tools available to
\n publishers, its usefulness has dissipated.\n<\/p>\n
\n responds to Googlebot’s HTTP requests. For example, if the server persistently returns HTTP
\n 500<\/code> status codes for a range of URLs, Googlebot will automatically, and almost
\n immediately slow down crawling. Similarly, Googlebot slows down automatically if the response time
\n for requests gets significantly longer. If you do experience unusually heavy crawling that your
\n site can’t manage on its own, refer to this
\n help article<\/a>.\n<\/p>\n
\n for the new limits to be applied on crawling. Fortunately though, site owners rarely had to resort
\n to using the tool, and those who have, in many cases set the crawling speed to the bare minimum.
\n With the deprecation of the crawl limiter tool, we’re also setting the minimum crawling speed to a
\n lower rate, comparable to the old crawl rate limits. This means that we effectively continue
\n honoring the settings that some site owners have set in the past if the Search interest is low,
\n and our crawlers don’t waste the site’s bandwidth.\n<\/p>\n
\n simple for users, we’ll be deprecating this tool in Search Console. We are keeping the Googlebot
\n report form<\/a> for reporting unusual
\n Googlebot activities and for emergency cases, but keep in mind that the fastest way to reduce
\n crawl rate is to instruct Googlebot through server responses as detailed in
\n our documentation<\/a>.\n<\/p>\n
\n Google Search Central Community<\/a>.\n<\/p>\n