Crawling and indexingOverview of crawling and indexing topics
Crawling and indexing help your site rank in search results. Browse topics to understand what it means for Google to…
Crawling and indexing
Crawling and indexingFile types indexable by Google
Google can index the content of most types of pages and files. Explore a list of the most common file…
Crawling and indexingURL structure best practices for Google Search
It's a good idea to keep your site's URL structure as simple as possible. Learn more about the URL format…
Crawling and indexingLink best practices for Google
Google uses links as a signal when determining the relevancy of pages and to find new pages to crawl. Learn…
SitemapsLearn about sitemaps
A sitemap provides information that helps Google more intelligently crawl your site. Discover how a sitemap works and determine if…
Build and submit a sitemap
Google supports several sitemap formats. Follow this guide to learn about formats, how to build a sitemap, and how to…
SitemapsManage your sitemaps with a sitemap index file
You can submit multiple sitemaps at once with a sitemaps index file. Learn how to manage large sitemaps.
Sitemap extensionsImage sitemaps
Image sitemaps help Google discover images. Follow these best practices on how to create an image sitemap and review image…
Sitemap extensionsNews sitemaps
Learn how to create and submit a News sitemap by reviewing the requirements, example entries, and news-specific tag definitions.
Sitemap extensionsVideo sitemaps and alternatives
Video sitemaps help Google display videos in Search. Review these guidelines to learn more and browse video xml sitemap examples.
Sitemap extensionsHow to combine sitemap extensions
Learn how to combine different kinds of sitemap extensions in one single XML sitemap.
Crawler managementAsk Google to recrawl your URLs
If you've recently added or made changes to a page on your site, you can request that Google re-index it…
Crawler managementTroubleshoot Google Search crawling errors
Learn how to troubleshoot crawling errors.
Google crawlersGooglebot
Googlebot is the generic name of the web crawler used by Google Search. Discover what Googlebot is, how it accesses…
robots.txtIntroduction to robots.txt
Robots.txt is used to manage crawler traffic. Explore this robots.txt introduction guide to learn what robot.txt files are and how…
CanonicalizationWhat is canonicalization
Learn what canonicalization is, how Google chooses a canonical URL, and whether it matters for your site.
How to specify a canonical URL with rel=”canonical” and other methods
When a site has duplicate content, Google chooses the canonical URL. Learn more about canonical URLs and how to consolidate…
CanonicalizationFix canonicalization issues
Learn how to debug the most common canonicalization issues in Google Search.
Crawling and indexingMobile site and mobile-first indexing best practices
Discover what Google mobile-first indexing is and explore best practices designed to improve user experience in Google Search.
AMPAbout AMP on Google Search
Google Search indexes AMP pages just like other web pages. Explore this page for commonly asked questions and guidelines for…
AMPEnhance AMP content in Google Search
You can enhance AMP content in Google Search in a few short steps. Review our tips for how to create…
AMPValidate your AMP content
Explore this guide to learn how to validate AMP content, how to fix AMP errors, and for more AMP resources.
AMPRemove your AMP pages from Google Search
Discover the different options for removing all versions of your AMP pages from Google Search, including AMP and non-AMP pages.
JavaScriptUnderstand the JavaScript SEO basics
Discover how Google Search processes JavaScript and explore best practices for improving JavaScript web apps for Google Search.
JavaScriptFix Search-related JavaScript problems
Read this guide to learn how you can identify and fix Search-related JavaScript problems that may be blocking your page…
JavaScriptFix lazy-loaded content
Lazy loading is a common website performance and UX best practice. Learn how to test and fix lazy loaded content…
JavaScriptDynamic rendering as a workaround
Dynamic rendering was a workaround for websites experiencing issues with JavaScript-powered content.
Page and content metadataUse valid HTML to specify page metadata
Using valid page metadata ensures that Google can process the HTML markup of your pages. Learn which elements are valid…
meta tags and attributes that Google supports
Google supports both page-level meta tags and tag attributes. Explore a list of the special tags that Google Search can…
Meta tagsRobots meta tag, data-nosnippet, and X-Robots-Tag specifications
Learn how to add robots meta tags and read how page and text-level settings can be used to adjust how…
Meta tagsBlock Search indexing with noindex
A noindex tag can block Google from indexing a page so that it won't appear in Search results. Learn how…
Page and content metadataQualify your outbound links to Google
You might want to tell Google your relationship with outbound links on your site. Discover how to qualify outbound links…
RemovalsControl what you share with Google
You can control what content Google sees on your site and what is shown in search results. Discover the main…
Remove a page hosted on your site from Google
If you need to remove a page that's hosted on your website from Google Search, you can submit a request…
RemovalsRemove images hosted on your site from search results
A robots.txt file can prevent images from appearing in search results. Discover how to remove and hide images on your…
RemovalsKeep redacted information out of Google Search
Learn best practices for how to redact information from documents that you don't want to be indexed and made discoverable…
Site moves and changesRedirects and Google Search
Learn about the different types of redirects, how Google can interpret them, and how they can be useful in Google…
Changing your hosting
Discover how to minimize the impact on your Google Search performance when transferring your site's hosting infrastructure without changing your…
How to move a site
Learn how to change the URLs of existing site pages, including domain name changes. Explore moving a website with little…
Minimize A/B testing impact in Google Search
Discover how to ensure A/B testing has minimal impact on Search by following website testing best practices for AB testing,…
Site moves and changesTemporarily pause or disable a website
If you are considering closing your business website, learn how to pause your site temporarily or disable it completely with…
Crawling and indexingDiagnosing Crawl Errors
Reading what Search Console reports, and separating errors that matter from ones that are working as intended.
Crawling and indexingGoogle Crawlers and How to Verify Them
Which user agents Google operates, and how to confirm a request claiming to be Googlebot really is.
Crawling and indexingHow Google Interprets robots.txt
Matching rules, wildcards, precedence and the size limit, the details that decide whether your file does what you think.
Crawling and indexingMobile-First Indexing
Google indexes the mobile version. What that means for content parity and markup.
Crawling and indexingRemoving Outdated Content You Do Not Control
When the page has changed or gone but Google still shows the old version.
Crawling infrastructureAPIs-Google user agent
APIs-Google is the user agent used by Google APIs to deliver push notifications. Explore this guide to learn how APIs-Google…
Crawling infrastructureAuthenticate requests with Web Bot Auth (experimental)
Learn what Web Bot Auth is, the current status of Google's implementation, and how you can implement verification during Google's…
Crawling infrastructureChangelog
Track major updates to Google's crawling documentation, including changes to Google's list of user agents, crawlers, and user-triggered fetchers.
Crawling infrastructureDebug network and DNS errors for Google’s crawlers
See how different network and DNS errors can affect Google's crawlers and how to debug network and DNS errors.