Yandex now crawls AJAX sites by fetching the original URLs and executing JavaScript on them. Two older mechanisms are formally retired, and sites still using them are working against the crawler.
What is no longer supported
meta name="fragment" content="!"— ignored. The robot indexes the original page instead. The companion?_escaped_fragment_=HTML-version scheme is therefore dead weight.- Hash-bang (
#!) URLs — Yandex recommends abandoning the approach entirely.
The migration
- Изменение sitemap links so they contain no
#. - Replace fragment-based links in the interface with real paths, using the History API.
Why this matters more than it sounds
Yandex states the rule plainly: when looking for links on AJAX pages, the bot only considers URLs in the href attribute, and fragments are not followed. Its own example is worth internalising — a navigation of <a href="#/clothes"> and <a href="#/shoes"> yields no discoverable pages at all. The same navigation written as /clothes and /shoes with History API routing is fully crawlable.
This is the difference between a single-page application that indexes and one that does not, and it is a routing decision made early in development by someone who is not thinking about search.
What to check on an existing application
- Are internal links real
hrefvalues with real paths, or are they click handlers on non-anchor elements? The second case is invisible to the crawler. - Does each route return a complete document when requested directly, or only when reached through the application shell?
- Does the sitemap contain the same URLs the router actually serves?
The direction of travel
The retirement of the fragment scheme means there is no special path for JavaScript applications any more. They are indexed as ordinary sites, which means they have to behave like ordinary sites: one URL per view, reachable directly, linked with anchors.
Как мы это применяем
The href-only rule is the fact worth handing to any front-end team before they choose a router. We have audited applications whose entire catalogue was unreachable because navigation was built on click handlers and fragments — the site worked perfectly for people and had one indexable page. Fixing it is a routing change, which is cheap early and expensive after launch.
Связанные услуги