Indexing

Why are pages excluded from the search?

Ask AI about this page

3 min read

Every excluded page carries a stated reason, listed under Indexing → Searchable pages → Excluded pages. Yandex is unusually forthcoming here — Google shows nothing comparable — and the reason is the diagnosis, so the list should be read by status rather than by URL.

The exclusion reasons

  • Low-value or low-demand — the algorithm chose not to include it. Not a violation.
  • Download or processing error (3XX, 4XX, 5XX) — verify with the server response check; confirm the page is in the sitemap and that no directive is blocking it.
  • Blocked in robots.txt or by noindex — remove the prohibition. If you did not add it, ask the hosting provider or registrar, and check that the domain has not lapsed.
  • The page redirects the robot elsewhere — confirm the redirect is intended.
  • Duplicate of another page.
  • Not canonical — the page points elsewhere with rel="canonical". Confirm that is deliberate.
  • Address recognised as an alternate address — the mirror grouping absorbed it; ungrouping is a separate procedure.
  • Violations found on the site — check Security and violations.

Two questions that come up on every audit

“The page opens fine in my browser.” Yandex gives the reason directly: the headers the robot sends differ from the browser’s, so a page can serve a browser and fail the robot. Also, a page excluded for a download error leaves the list only once it is available to the robot again — check the server response, and if it returns 200 OK, wait for the next crawl.

“These pages do not exist any more.” The list includes URLs the robot reached but did not index, including URLs that no longer exist. They leave only when they are unavailable to the robot for a period and nothing links to them, internally or externally. Yandex repeats the reassurance here: excluded pages do not affect the site’s position.

The pattern behind most large exclusion lists

Yandex names it explicitly: products differing only by colour, size or configuration; pagination pages; product selection and comparison pages; image pages with no text. Automatically generated title and description pairs that duplicate each other make this worse. On a catalogue site this is usually one template decision, not thousands of separate problems.

After fixing

Submit the page for reindexing rather than waiting. The robot keeps visiting excluded pages and a special algorithm re-evaluates them before each index update, so a corrected page can return within about two weeks.

Alien Road

Comment nous l'appliquons

We work this list grouped by status and sorted by count, because a hundred exclusions almost always reduce to two template causes. The status that most often turns out to be a genuine mistake is “not canonical”: a canonical tag generated by a plugin pointing every filtered page at the category root, quietly removing pages the client intended to rank. It is invisible in a browser and obvious in this report.

Services associés

Partager

© Copyright 2026 Alien Road. All rights reserved.