{"id":16573,"date":"2026-09-02T11:36:55","date_gmt":"2026-09-02T08:36:55","guid":{"rendered":"https:\/\/alienroad.com\/google-bilgi-bankasi\/robots-meta-tag\/"},"modified":"2026-09-02T16:46:02","modified_gmt":"2026-09-02T13:46:02","slug":"robots-meta-tag","status":"publish","type":"ar_kb","link":"https:\/\/alienroad.com\/google-bilgi-bankasi\/robots-meta-tag\/","title":{"rendered":"The Robots Meta Tag and X-Robots-Tag"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Where <code>robots.txt<\/code> controls crawling for the whole site, the robots meta tag controls indexing and serving for a single page. It comes in two forms: an HTML <code>&lt;meta&gt;<\/code> element, and an <code>X-Robots-Tag<\/code> HTTP header for non-HTML files.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The main directives<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><code>noindex<\/code>, do not show this page in results.<\/li><li><code>nofollow<\/code>, do not follow links on this page.<\/li><li><code>none<\/code>, equivalent to <code>noindex, nofollow<\/code>.<\/li><li><code>nosnippet<\/code>, no text snippet.<\/li><li><code>max-snippet:[n]<\/code>, <code>max-image-preview:[setting]<\/code>, <code>max-video-preview:[n]<\/code>.<\/li><li><code>noarchive<\/code>, <code>notranslate<\/code>, <code>noimageindex<\/code>.<\/li><li><code>unavailable_after:[date]<\/code>, drop from the index after a date.<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">The combination that fails<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Blocking a URL in <code>robots.txt<\/code> and adding <code>noindex<\/code> to the page. Google never fetches the page, never reads the tag, and the URL can remain indexed. To remove a page: allow crawling, add <code>noindex<\/code>, wait for a recrawl, and only then block it if you want to.<\/p>\n\n\n\n<div class=\"ar-kb-tavsiye\"><p><strong>Alien Road recommends:<\/strong> use <code>data-nosnippet<\/code> on specific elements rather than <code>nosnippet<\/code> on the whole page when you only want to keep prices or personal details out of a snippet. Suppressing the entire snippet reliably reduces click-through; suppressing one paragraph does not.<\/p><\/div>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Per-page indexing control, the directives available and the one combination that never works.<\/p>\n","protected":false},"menu_order":20,"template":"","meta":{"footnotes":""},"ar_kb_kategori":[506],"ar_kb_etiket":[530,578,579],"class_list":["post-16573","ar_kb","type-ar_kb","status-publish","hentry","ar_kb_kategori-page-metadata","ar_kb_etiket-indexing","ar_kb_etiket-noindex","ar_kb_etiket-robots"],"_links":{"self":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/16573","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb"}],"about":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/types\/ar_kb"}],"version-history":[{"count":1,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/16573\/revisions"}],"predecessor-version":[{"id":23689,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb\/16573\/revisions\/23689"}],"wp:attachment":[{"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/media?parent=16573"}],"wp:term":[{"taxonomy":"ar_kb_kategori","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_kategori?post=16573"},{"taxonomy":"ar_kb_etiket","embeddable":true,"href":"https:\/\/alienroad.com\/wp-json\/wp\/v2\/ar_kb_etiket?post=16573"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}