{"id":28522,"date":"2026-09-08T21:56:08","date_gmt":"2026-09-08T18:56:08","guid":{"rendered":"https:\/\/alienroad.com\/yandex-bilgi-bankasi\/how-do-i-prohibit-indexing-of-the-site-or-pages\/"},"modified":"2026-09-09T00:47:43","modified_gmt":"2026-09-08T21:47:43","slug":"how-do-i-prohibit-indexing-of-the-site-or-pages","status":"publish","type":"ar_ykb","link":"https:\/\/alienroad.com\/yandex-bilgi-bankasi\/how-do-i-prohibit-indexing-of-the-site-or-pages\/","title":{"rendered":"How do I prohibit indexing of the site or pages?"},"content":{"rendered":"<p>Three instruments hide content from indexing, and they operate at different levels: the whole site or a section, an individual page, or part of a page&#8217;s text.<\/p>\n<h2>Site, section or page<\/h2>\n<ul>\n<li><strong><code>Disallow<\/code><\/strong> in <code>robots.txt<\/code>.<\/li>\n<li><strong>The <code>robots<\/code> meta tag<\/strong> with <code>noindex<\/code> o <code>none<\/code>.<\/li>\n<li><strong>Authorisation<\/strong> \u2014 and Yandex specifically recommends this for hiding a home page, with a reason worth remembering: <em>a home page blocked in <code>robots.txt<\/code> can still appear in results if other sites link to it<\/em>. Blocking is not concealment.<\/li>\n<\/ul>\n<p>For anything genuinely restricted, combine authorisation with a server response of <strong>404, 403 or 410<\/strong> for unauthorised requests, so the page does not merely refuse to render but reports itself as unavailable.<\/p>\n<h2>Part of a page&#8217;s text<\/h2>\n<p>The <code>noindex<\/code> <em>element<\/em> \u2014 a Yandex extension, distinct from the meta tag:<\/p>\n<pre><code>&lt;noindex&gt;text that should not be indexed&lt;\/noindex&gt;<\/code><\/pre>\n<p>Two rules make it dangerous if misused:<\/p>\n<ul>\n<li><strong>If the closing tag is missing, the entire remaining content of the page is treated as hidden.<\/strong> One unclosed tag can remove a whole page&#8217;s text from the index.<\/li>\n<li><strong>Nested tags are ignored after the first closing tag<\/strong>, so nesting does not work as written.<\/li>\n<\/ul>\n<p>For valid markup, the comment form is accepted: <code>&lt;!--noindex--&gt;\u2026&lt;!--\/noindex--&gt;<\/code>.<\/p>\n<h2>Choosing between them<\/h2>\n<p>Ask what should happen if someone types the URL directly. If the answer is &#8220;they should see it, but it should not rank&#8221; \u2014 <code>noindex<\/code>. If it is &#8220;the robot should not waste time here&#8221; \u2014 <code>Disallow<\/code>. If it is &#8220;nobody should see this&#8221; \u2014 authorisation, with an appropriate status code. The three are not interchangeable, and the most common mistake is using the second for the third.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Three instruments hide content from indexing, and they operate at different levels: the whole site or a section, an individual page, or part of a page&#8217;s text. Site, section or page Disallow in robots.txt. The robots meta tag with noindex o none. Authorisation \u2014 and Yandex specifically recommends this for hiding a home page, with [&hellip;]<\/p>\n","protected":false},"menu_order":98,"template":"","meta":{"footnotes":""},"ar_ykb_kategori":[764],"ar_ykb_etiket":[],"class_list":["post-28522","ar_ykb","type-ar_ykb","status-publish","has-post-thumbnail","hentry","ar_ykb_kategori-site-indexing"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb\/28522","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_ykb"}],"version-history":[{"count":2,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb\/28522\/revisions"}],"predecessor-version":[{"id":29140,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb\/28522\/revisions\/29140"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media\/28904"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=28522"}],"wp:term":[{"taxonomy":"ar_ykb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb_kategori?post=28522"},{"taxonomy":"ar_ykb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_ykb_etiket?post=28522"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}