{"id":25710,"date":"2026-09-06T00:50:31","date_gmt":"2026-09-05T21:50:31","guid":{"rendered":"https:\/\/alienroad.com\/google-bilgi-bankasi\/feedfetcher\/"},"modified":"2026-09-06T00:50:31","modified_gmt":"2026-09-05T21:50:31","slug":"feedfetcher","status":"publish","type":"ar_kb","link":"https:\/\/alienroad.com\/google-bilgi-bankasi\/feedfetcher\/","title":{"rendered":"Feedfetcher"},"content":{"rendered":"

\n Feedfetcher is how Google crawls RSS or Atom feeds for
\n Google News<\/a>
\n and WebSub<\/a>.
\n Feedfetcher stores and periodically refreshes feeds that are requested by users of an app or
\n service. Only podcast feeds get indexed in Google Search; however, if a feed doesn’t follow the
\n Atom<\/a> o
\n RSS<\/a> specification, it
\n may still be indexed. Here are some answers to the most commonly asked questions about how this
\n user-controlled feed grabber works.\n<\/p>\n

How do I request that Google not retrieve some or all of my site’s feeds?<\/h2>\n

\n When users add a service or app that uses Feedfetcher data, Google’s Feedfetcher attempts to
\n obtain the content of the feed in order to display it. Since Feedfetcher requests come from
\n explicit action by human users, and not from automated crawlers, Feedfetcher ignores robots.txt
\n rules.\n<\/p>\n

\n If your feed is publicly available, Google can’t restrict users from accessing it. One
\n solution is to configure your site to serve a 404<\/code>, 410<\/code>, or other error
\n status message to Feedfetcher-Google<\/code> user agent.\n<\/p>\n

\n If your feed is provided by a blog or site hosting service, work directly with that service to
\n restrict access to your feed.\n<\/p>\n

How often will Feedfetcher retrieve my feeds?<\/h2>\n

\n Feedfetcher shouldn’t retrieve feeds from most sites more than once every hour on average. Some
\n frequently updated sites may be refreshed more often. Note, however, that due to network delays,
\n it’s possible that Feedfetcher may briefly appear to retrieve your feeds more frequently.\n<\/p>\n

Why is Feedfetcher downloading information from my “secret” web server?<\/h2>\n

\n Feedfetcher retrieves feeds at the request of services or apps installed by users. It is
\n possible that the request came from a user who knows about your “secret” server or typed it in
\n by mistake.\n<\/p>\n

Why isn’t Feedfetcher obeying my robots.txt file?<\/h2>\n

\n Feedfetcher retrieves feeds only after users have explicitly started a service or app that
\n requests data from the feed. Feedfetcher behaves as a direct agent of the human user, not as a
\n robot, so it ignores robots.txt entries. Since Feedfetcher acts as an agent for multiple
\n users, it conserves bandwidth by making requests for common feeds only once for all users who
\n requested the feed through an app or service. The common feeds are
\n RSS<\/a> and
\n Atom<\/a>.\n<\/p>\n

\n You can prevent Feedfetcher from crawling your site by configuring your server to serve a
\n 404<\/code>, 410<\/code>, or other error status message to the
\n Feedfetcher-Google<\/code> user agent.\n<\/p>\n

Why are there visits from multiple machines at Google.com, all with user-agent Feedfetcher?<\/h2>\n

\n Feedfetcher was designed to be distributed on several machines to improve performance and scale as
\n the web grows. To cut down on bandwidth usage, the machines used are often located near the sites
\n that they’re retrieving in the network.\n<\/p>\n

\n Can you tell me the IP addresses from which Feedfetcher makes requests so that I can filter my
\n logs?
\n<\/h2>\n

\n The IP addresses used by Feedfetcher are included in the
\n user-triggered-fetchers-google.json<\/a>
\n object.\n<\/p>\n

Why is Feedfetcher downloading the same page on my site multiple times?<\/h2>\n

\n In general, Feedfetcher only downloads one copy of each file from your site during a given feed
\n retrieval. Very occasionally, the machines are stopped and restarted, which may cause it to again
\n retrieve pages that it’s recently visited.\n<\/p>\n

My Feedfetcher question isn’t answered here. Where can I get more help?<\/h2>\n

\n If you’re still having trouble, try posting your question in the Search Central
\n forum<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Feedfetcher is how Google grabs site feeds. Explore this guide to learn what Google Feedfetcher is and how this user-controlled feed grabber works.<\/p>\n","protected":false},"menu_order":7000,"template":"","meta":{"footnotes":""},"ar_kb_kategori":[708],"ar_kb_etiket":[],"class_list":["post-25710","ar_kb","type-ar_kb","status-publish","has-post-thumbnail","hentry","ar_kb_kategori-crawling-infrastructure-crawling-and-indexing"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/25710","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_kb"}],"version-history":[{"count":0,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/25710\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media\/27676"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=25710"}],"wp:term":[{"taxonomy":"ar_kb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_kategori?post=25710"},{"taxonomy":"ar_kb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_etiket?post=25710"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}