Updating Our Search Quality Rating Guidelines

Thursday, November 19, 2015

Developing algorithmic changes to search involves a
process of experimentation.
Part of that experimentation is having evaluators—people who assess the quality of Google’s search
results—give us feedback on our experiments. Ratings from evaluators do not determine individual
site rankings, but are used help us understand our experiments. The evaluators base their ratings
on guidelines we give them; the guidelines reflect what Google thinks search users want.

In 2013, we published our human rating guidelines to provide transparency on how Google works and
to help webmasters understand what Google looks for in web pages. Since that time, a lot has
changed: notably, more people have smartphones than ever before and more searches are done on
mobile devices today than on computers.

We often make changes to the guidelines as our understanding of what users wants evolves, but we
haven’t shared an update publicly since then. However, we recently completed a major revision of
our
rater guidelines
to adapt to this mobile world, recognizing that people use Search differently when they carry
internet-connected devices with them all the time.

This is not the final version of our rater guidelines. The guidelines will continue to evolve as
Search, and how people use it, changes. We won’t be updating the public document with every
change, but we will try to publish big changes to the guidelines periodically.

We expect our phones and other devices to do a lot, and we want Google to continue giving users
the answers they’re looking for—fast!

Posted by Mimi Underwood, Sr. Program Manager, Search Growth and Analysis