{"id":24660,"date":"2012-03-12T00:00:00","date_gmt":"2012-03-12T00:00:00","guid":{"rendered":"https:\/\/alienroad.com\/google-bilgi-bankasi\/crawl-errors-the-next-generation\/"},"modified":"2012-03-12T00:00:00","modified_gmt":"2012-03-12T00:00:00","slug":"crawl-errors-the-next-generation","status":"publish","type":"ar_kb","link":"https:\/\/alienroad.com\/google-bilgi-bankasi\/crawl-errors-the-next-generation\/","title":{"rendered":"Crawl Errors: The Next Generation"},"content":{"rendered":"

Monday, March 12, 2012<\/p>\n

\n Crawl errors is one of the most popular features in Webmaster Tools, and today we’re rolling out
\n some very significant enhancements that will make it even more useful.\n<\/p>\n

\n We now detect and report many new types of errors. To help make sense of the new data, we’ve split
\n the errors into two parts: site errors<\/em> and URL errors<\/em>.\n<\/p>\n

Site Errors<\/h2>\n

\n Site errors are errors that aren’t specific to a particular URL\u2014they affect your entire site.
\n These include DNS resolution failures, connectivity issues with your web server, and problems
\n fetching your robots.txt file. We used to report these errors by URL, but that didn’t make a lot
\n of sense because they aren’t specific to individual URLs\u2014in fact, they prevent Googlebot from even
\n requesting a URL! Instead, we now keep track of the failure rates for each type of site-wide
\n error. We’ll also try to send you alerts when these errors become frequent enough that they
\n warrant attention.\n<\/p>\n

\n \"View
\n<\/figure>\n

\n Furthermore, if you don’t have (and haven’t recently had) any problems in these areas, as is the
\n case for many sites, we won’t bother you with this section. Instead, we’ll just show you some
\n friendly check marks to let you know everything is hunky-dory.\n<\/p>\n

\n \"Site
\n<\/figure>\n

URL errors<\/h2>\n

\n URL errors are errors that are specific to a particular page. This means that when Googlebot tried
\n to crawl the URL, it was able to resolve your DNS, connect to your server, fetch and read your
\n robots.txt file, and then request this URL, but something went wrong after that. We break the URL
\n errors down into various categories based on what caused the error. If your site serves up Google
\n News or mobile (CHTML\/XHTML) data, we’ll show separate categories for those errors.\n<\/p>\n

\n \"URL
\n<\/figure>\n

Less is more<\/h2>\n

\n We used to show you at most 100,000 errors of each type. Trying to consume all this information
\n was like drinking from a firehose, and you had no way of knowing which of those errors were
\n important (your home page is down) or less important (someone’s personal site made a typo in a link
\n to your site). There was no realistic way to view all 100,000 errors\u2014no way to sort, search, or
\n mark your progress. In the new version of this feature, we’ve focused on trying to give you only
\n the most important errors up front. For each category, we’ll give you what we think are the 1000
\n most important and actionable errors. You can sort and filter these top 1000 errors, let us know
\n when you think you’ve fixed them, and view details about them.\n<\/p>\n

\n \"Sorting
\n<\/figure>\n

\n Some sites have more than 1000 errors of a given type, so you’ll still be able to see the total
\n number of errors you have of each type, as well as a graph showing historical data going back 90
\n days. For those who worry that 1000 error details plus a total aggregate count will not be enough,
\n we’re considering adding programmatic access (an API) to allow you to download every last error
\n you have, so please give us feedback if you need more.\n<\/p>\n

\n We’ve also removed the list of pages blocked by robots.txt, because while these can sometimes be
\n useful for diagnosing a problem with your robots.txt file, they are frequently pages you
\n intentionally<\/em> blocked. We really wanted to focus on errors, so look for information about
\n roboted URLs to show up soon in the “Crawler access” feature under “Site configuration”.\n<\/p>\n

Dive into the details<\/h2>\n

\n Clicking on an individual error URL from the main list brings up a detail pane with additional
\n information, including when we last tried to crawl the URL, when we first noticed a problem, and a
\n brief explanation of the error.\n<\/p>\n

\n \"Detail
\n<\/figure>\n

\n From the details pane you can click on the link for the URL that caused the error to see for
\n yourself what happens when you try to visit it. You can also mark the error as “fixed” (more on
\n that later!), view help content for the error type, list Sitemaps that contain the URL, see other
\n pages that link to this URL, and even have Googlebot fetch the URL right now, either for more
\n information or to double-check that your fix worked.\n<\/p>\n

\n \"Panel
\n<\/figure>\n

Take action!<\/h2>\n

\n One thing we’re really excited about in this new version of the Crawl errors feature is that you
\n can really focus on fixing what’s most important first. We’ve ranked the errors so that those at
\n the top of the priority list will be ones where there’s something you can do, whether that’s
\n fixing broken links on your own site, fixing bugs in your server software, updating your sitemaps
\n to prune dead URLs, or adding a 301<\/code> redirect to get users to the “real” page. We determine this
\n based on a multitude of factors, including whether or not you included the URL in a Sitemap, how
\n many places it’s linked from (and if any of those are also on your site), and whether the URL has
\n gotten any traffic recently from search.\n<\/p>\n

\n Once you think you’ve fixed the issue (you can test your fix by fetching the URL as Googlebot),
\n you can let us know by marking the error as “fixed” if you are a
\n user with full access permissions<\/a>.
\n This will remove the error from your list. In the future, the errors you’ve marked as fixed won’t
\n be included in the top errors list, unless we’ve encountered the same error when trying to
\n re-crawl a URL.\n<\/p>\n

\n \"Panel
\n<\/figure>\n

\n We’ve put a lot of work into the new Crawl errors feature, so we hope that it will be very useful
\n to you. Let us know what you think and if you have any suggestions, please visit our
\n forum<\/a>!\n<\/p>\n

\n Written by
\n Kurt Dresner<\/a>,
\n Webmaster Tools team<\/p>\n","protected":false},"excerpt":{"rendered":"

Monday, March 12, 2012 Crawl errors is one of the most popular features in Webmaster Tools, and today we’re rolling out some very significant enhancements that will make it even more useful. We now detect and report many new types of errors. To help make sense of the new data, we’ve split the errors into […]<\/p>\n","protected":false},"menu_order":84589,"template":"","meta":{"footnotes":""},"ar_kb_kategori":[665],"ar_kb_etiket":[],"class_list":["post-24660","ar_kb","type-ar_kb","status-publish","has-post-thumbnail","hentry","ar_kb_kategori-blog"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/24660","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_kb"}],"version-history":[{"count":0,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/24660\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media\/26925"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=24660"}],"wp:term":[{"taxonomy":"ar_kb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_kategori?post=24660"},{"taxonomy":"ar_kb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_etiket?post=24660"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}