Taking feeds out of our web search results

Tuesday, December 18, 2007

As a webmaster, you may have been concerned about your RSS/Atom feeds crowding out their
associated HTML pages in Google’s search results. By serving feeds, we could cause a poor user
experience:

1. Feeds increase the likelihood that users see duplicate search results.
2. Users clicking on a feed may miss valuable content available only in the HTML page.

To address these concerns, we prevent feeds from being returned in Google’s search results, with
the exception of podcasts (feeds with multimedia enclosures). We continue to allow podcasts,
because we noticed a significant number of them are standalone documents (that is, no HTML page
has the same content) or they have more complete item descriptions than the associated HTML page.
However, if, as a webmaster, you’d like your podcasts to be excluded from Google’s search results
(for example, if you have a vlog, its feed is probably a podcast), you can use
Yahoo’s spec for noindex feeds.
If you use FeedBurner, making your
podcast noindex is as simple as checking a box (“Noindex” under the “Publicize” tab).

As a user, you may ask yourself whether Google has a way to search for feeds. The answer is yes;
both Google Reader and
iGoogle allow searching for feeds to
subscribe to.

We’re aware that there are a few non-podcast feeds out there with no associated HTML pages, and
thus removing these feeds for now from the search results might be less than ideal. We remain open
to other feedback on how to improve the handling of feeds, and especially welcome your comments
and questions in the
Crawling, Indexing and Ranking
subtopic of our Webmaster Help Group.

Written by Bogdan Stănescu, Software Engineer