Tuesday, March 06, 2007
Search engine robots, including our very own Googlebot, are incredibly polite. They work
hard to respect your every wish regarding what pages they should and should not crawl. How can
they tell the difference? You have to tell them, and you have to speak their language, which is
an industry standard called the
Robots Exclusion Protocol.
Dan Crow has written about this on the Google Blog recently, including an introduction to setting
up your own rules for robots and a description of some of the more advanced options. His first
two posts in the series are:
Stay tuned for the next installment.
Stay tuned for the next installment.
While we’re on the topic, I’d also like to point you to the
robots section of our help center
and our earlier posts on this topic:
Bunu necə tətbiq edirik
We archive Search Central announcements because client questions often trace back to a change that was announced years ago. Reading the original is faster than reconstructing it from second-hand summaries.
Əlaqəli xidmətlər