The console’s robots.txt analyser reports errors and warnings, several of which are Yandex extensions — rules the file may break in Yandex without breaking anywhere else. That distinction is the reason this reference matters: a file that validates against Google’s parser can still be rejected here.
Errors, with the Yandex-specific ones marked
- Rule does not start with / or * (Yandex extension) — a rule may begin only with those characters.
- Multiple
User-agent: *rules — only one such block is allowed. - File size limit exceeded (Yandex) — the file exceeds 2,048 rules. Generated robots.txt files on large catalogues hit this.
- No User-agent directive before a rule — rules must follow a
User-agentline. Yandex adds the likely cause: an empty line afterUser-agent, which terminates the block. The file looks correct and the rules below it apply to nobody. - Rule too long (Yandex) — over 1,024 characters.
- Invalid Sitemap URL (Yandex) — the sitemap must be given in full, with protocol. A relative path is a common and silent mistake.
- Invalid Clean-param format (Yandex) — one or more parameters separated by
&, then a space, then the path prefix.
Warnings
Illegal character (anything special other than * and $); unknown directive (not in Yandex’s rules — often a valid directive for another engine, which is why this is a warning rather than an error); syntax error on a line that cannot be interpreted; and unknown error, which is a support case.
URL validation errors
When testing a specific URL against the file: a syntax error in the URL, or “this URL does not belong to your domain” — which usually means you pasted an alternate address of the site rather than the primary one. Given how Yandex handles mirrors, that mistake is easy to make and confusing to read.
How to use the analyser properly
Do not stop at “no errors”. Enter the URLs you believe should be crawled and the ones you believe should be blocked, and confirm the verdict for each. A syntactically perfect file that blocks the wrong directory reports no errors at all.
How we apply this
The two findings we hit most often are the empty line after User-agent, which silently detaches every rule beneath it, and a relative sitemap path that Yandex simply ignores. Neither shows up as a visible problem — the file is served, the syntax looks fine, and the rules do nothing. We always test specific URLs rather than trusting a clean parse, and we test them against the primary address, because the alternate one returns a confusing domain error.
Related services