{"id":25675,"date":"2026-09-06T00:47:47","date_gmt":"2026-09-05T21:47:47","guid":{"rendered":"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-http-status-codes-affect-googles-crawlers\/"},"modified":"2026-09-06T00:51:37","modified_gmt":"2026-09-05T21:51:37","slug":"how-http-status-codes-affect-googles-crawlers","status":"publish","type":"ar_kb","link":"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-http-status-codes-affect-googles-crawlers\/","title":{"rendered":"How HTTP status codes affect Google&#8217;s crawlers"},"content":{"rendered":"<p>\n  This page describes how different<br \/>\n  <a href=\"https:\/\/en.wikipedia.org\/wiki\/List_of_HTTP_status_codes\" class=\"external-link\">HTTP status codes<\/a>,<br \/>\n  impact Google&#8217;s ability to crawl your web content. We cover the top 20 status codes that Google<br \/>\n  encounters on the web. More exotic status<br \/>\n  codes, such as<br \/>\n  <a href=\"https:\/\/developer.mozilla.org\/docs\/Web\/HTTP\/Status\/418\" class=\"external-link\"><code>418 (I'm a teapot)<\/code><\/a>,<br \/>\n  aren&#8217;t covered.\n<\/p>\n<aside class=\"note\">\n  Experimental features of the supported protocols (HTTP and FTP) aren&#8217;t supported, unless<br \/>\n  otherwise stated.<br \/>\n<\/aside>\n<h2 id=\"http-status-codes\" tabindex=\"-1\">HTTP status codes<\/h2>\n<p>\n  HTTP status codes are generated by the server that&#8217;s hosting the site when it responds to a<br \/>\n  request made by a client, for example a browser or a crawler. Every HTTP status code has a<br \/>\n  different meaning, but often the outcome of the request is the same. For example, there are<br \/>\n  multiple status codes that signal redirection, but their outcome is the same.\n<\/p>\n<p>\n  Search Console generates error messages for status codes in the <code>4xx&mdash;5xx<\/code> range,<br \/>\n  and for failed redirections (<code>3xx<\/code>). If the server responded with a<br \/>\n  <code>2xx<\/code> status code, the content received in the response may be considered for<br \/>\n  indexing.\n<\/p>\n<aside class=\"note\">\n  For Google Search, an HTTP <code>2xx (success)<\/code> status code doesn&#8217;t guarantee indexing.<br \/>\n<\/aside>\n<p>\n  The following table contains the most encountered HTTP status codes by Google and an<br \/>\n  explanation how Google handles each status code.\n<\/p>\n<table class=\"responsive\">\n<tr>\n<th colspan=\"2\"><b>HTTP status codes<\/b><\/th>\n<\/tr>\n<tr>\n<td>\n<h3 id=\"2xx-success\" tabindex=\"-1\">\n      <code>2xx (success)<\/code><br \/>\n      <\/h3>\n<\/td>\n<td>\n<p>\n        Google considers the content for processing (for example, in the case of Google Search, for<br \/>\n        indexing). If the content suggests an error for Google Search,<br \/>\n        an empty page or an error message, Search Console will show a<br \/>\n        <a href=\"https:\/\/support.google.com\/webmasters\/answer\/7440203#zippy=%2Cerrors\" class=\"external-link\"><code>soft 404<\/code> error<\/a>.\n      <\/p>\n<table class=\"fixed\">\n<tr>\n<td width=\"30%\">\n            <code>200 (success)<\/code><\/td>\n<td rowspan=\"2\">\n<p>\n              Google passes on whatever it received to the next processing step (which is product specific).<br \/>\n              For Google Search, the next system is the indexing pipeline. The indexing systems<br \/>\n              may index the content, but that&#8217;s not guaranteed.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>201 (created)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>202 (accepted)<\/code><\/td>\n<td>\n<p>\n              Google waits for the content for a limited time, then passes on whatever it<br \/>\n              received to the next processing step (which is product specific). The timeout is user<br \/>\n              agent dependent, for example Googlebot Smartphone may have a different timeout than<br \/>\n              Googlebot Image.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>204 (no content)<\/code><\/td>\n<td>\n<p>\n              Google wasn&#8217;t able to receive any content and therefore can&#8217;t process it.\n            <\/p>\n<\/td>\n<\/tr>\n<\/table>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h3 id=\"3xx-redirection\" tabindex=\"-1\">\n      <code>3xx (redirection)<\/code><br \/>\n      <\/h3>\n<\/td>\n<td>\n<p>\n       By default, Google&#8217;s crawlers follow up to 10 redirect hops. However, specific products&#8217;<br \/>\n       crawlers may have different limits. For example, Googlebot generally follows 10 redirect<br \/>\n       hops when crawling for general web content, but Google Inspection Tools doesn&#8217;t follow redirects.\n      <\/p>\n<p>\n        Any content Google receives from the redirecting URL is ignored, and the final target<br \/>\n        URL&#8217;s content is processed instead. For robots.txt files, learn how<br \/>\n        <a href=\"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-google-interprets-the-robots-txt-specification\/#redirection\">Google handles a robots.txt that returns a <code>3xx<\/code> status code<\/a>.\n      <\/p>\n<table class=\"fixed\">\n<tr>\n<td width=\"30%\"><code>301 (moved permanently)<\/code><\/td>\n<td>\n<p>\n              Google follows the redirect, and Google systems use the redirect as a<br \/>\n              <b>strong<\/b> signal that the redirect target should be processed.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>302 (found)<\/code><\/td>\n<td rowspan=\"2\">\n<p>\n              By default, Google&#8217;s crawlers follow the redirect, and Google systems use the redirect as a<br \/>\n              <b>weak<\/b> signal that the redirect target should be processed. Other products may<br \/>\n              handle the redirect differently.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>303 (see other)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>304 (not modified)<\/code><\/td>\n<td>\n<p>\n              Google crawlers signal the next processing system that the content is the same as last time<br \/>\n              it was crawled. In the case of Google Search, the indexing pipeline may recalculate<br \/>\n              signals for the URL, but otherwise the status code has no effect on indexing.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>307 (temporary redirect)<\/code><\/td>\n<td>Equivalent to <code>302<\/code>.<\/td>\n<\/tr>\n<tr>\n<td><code>308 (moved permanently)<\/code><\/td>\n<td>Equivalent to <code>301<\/code>.<\/td>\n<\/tr>\n<\/table>\n<aside class=\"note\">\n        While Google treats these status codes the same way, keep in mind that<br \/>\n        they&#8217;re semantically different. Use the status code that&#8217;s appropriate for the redirect<br \/>\n        so other clients (for example, e-readers, other search engines) may benefit from it.<br \/>\n      <\/aside>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h3 id=\"4xx-client-errors\" tabindex=\"-1\">\n      <code>4xx (client errors)<\/code><br \/>\n      <\/h3>\n<\/td>\n<td>\n<p>\n        Google doesn&#8217;t use the content from URLs that return <code>4xx<\/code> status codes. If a<br \/>\n         URL was previously used but is now returning <code>4xx<\/code> status code, Google<br \/>\n         systems will stop using the URL over time. In the case of Google Search, Google doesn&#8217;t<br \/>\n         index URLs that return a <code>4xx<\/code> status code, and URLs that are already indexed<br \/>\n         and return a <code>4xx<\/code> status code are removed from the index.\n      <\/p>\n<p>\n        Any content Google receives from URLs that return a <code>4xx<\/code> status code is<br \/>\n        ignored.\n      <\/p>\n<table class=\"fixed\">\n<tr>\n<td width=\"30%\"><code>400 (bad request)<\/code><\/td>\n<td rowspan=\"6\" >\n<p>\n              Alle <code>4xx<\/code> errors, except <code>429<\/code>, are treated the same:<br \/>\n              Google crawlers inform the next processing system that the content doesn&#8217;t exist.\n            <\/p>\n<p>\n              In the case of Google Search, the indexing pipeline removes the URL from the index if<br \/>\n              it was previously indexed. Newly encountered <code>404<\/code> pages aren&#8217;t processed.<br \/>\n              The crawling frequency gradually decreases.\n            <\/p>\n<aside class=\"warning\">\n              Don&#8217;t use <code>401<\/code> and <code>403<\/code> status codes for limiting the<br \/>\n              crawl rate. The <code>4xx<\/code> status codes, except <code>429<\/code>, have no<br \/>\n              effect on crawl rate.<br \/>\n              <a href=\"https:\/\/alienroad.com\/google-bilgi-bankasi\/reduce-the-google-crawl-rate\/\">Learn how to limit your crawl rate<\/a>.<br \/>\n            <\/aside>\n<\/td>\n<\/tr>\n<tr>\n<td><code>401 (unauthorized)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>403 (forbidden)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>404 (not found)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>410 (gone)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>411 (length required)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>429 (too many requests)<\/code><\/td>\n<td>\n<p>\n              Google&#8217;s crawlers treat the <code>429<\/code> status code as a signal that the server is<br \/>\n              overloaded, and it&#8217;s considered a server error.\n            <\/p>\n<\/td>\n<\/tr>\n<\/table>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<h3 id=\"5xx-server-errors\" tabindex=\"-1\">\n      <code>5xx (server errors)<\/code><br \/>\n      <\/h3>\n<\/td>\n<td>\n<p>\n        <code>5xx<\/code> and <code>429<\/code> server errors prompt Google&#8217;s crawlers to<br \/>\n        temporarily slow down with crawling. For Google Search, already indexed URLs are preserved in the index,<br \/>\n        but eventually dropped.\n      <\/p>\n<p>\n        Any content Google receives from URLs that return a <code>5xx<\/code> status code is<br \/>\n        ignored. For robots.txt files, learn how<br \/>\n        <a href=\"https:\/\/alienroad.com\/google-bilgi-bankasi\/how-google-interprets-the-robots-txt-specification\/#server-error\">Google handles a robots.txt that returns a <code>5xx<\/code> status code<\/a>.\n      <\/p>\n<p>\n        Once the server starts responding with a <code>2xx<\/code> status code, Google gradually<br \/>\n        increases the crawl rate for the site.\n      <\/p>\n<table class=\"fixed\">\n<tr>\n<td width=\"30%\"><code>500 (internal server error)<\/code><\/td>\n<td rowspan=\"3\">\n<p>\n              Google decreases the crawl rate for the site. The decrease in crawl rate is<br \/>\n              proportionate to the number of individual URLs that are returning a server error.<br \/>\n              For Google Search, Google&#8217;s indexing pipeline removes from the index URLs that persistently return a<br \/>\n              server error.\n            <\/p>\n<\/td>\n<\/tr>\n<tr>\n<td><code>502 (bad gateway)<\/code><\/td>\n<\/tr>\n<tr>\n<td><code>503 (service unavailable)<\/code><\/td>\n<\/tr>\n<\/table>\n<\/td>\n<\/tr>\n<\/table>\n","protected":false},"excerpt":{"rendered":"<p>See how different HTTP status codes can affect Google&#8217;s crawlers.<\/p>\n","protected":false},"menu_order":7000,"template":"","meta":{"footnotes":""},"ar_kb_kategori":[708],"ar_kb_etiket":[],"class_list":["post-25675","ar_kb","type-ar_kb","status-publish","has-post-thumbnail","hentry","ar_kb_kategori-crawling-infrastructure-crawling-and-indexing"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/25675","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_kb"}],"version-history":[{"count":1,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/25675\/revisions"}],"predecessor-version":[{"id":25677,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/25675\/revisions\/25677"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media\/27670"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=25675"}],"wp:term":[{"taxonomy":"ar_kb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_kategori?post=25675"},{"taxonomy":"ar_kb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_etiket?post=25675"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}