{"id":17669,"date":"2021-03-09T09:25:43","date_gmt":"2021-03-09T09:25:43","guid":{"rendered":"http:\/\/wiki2.plumserver.com\/?post_type=ht_kb&#038;p=17669"},"modified":"2021-07-22T08:16:33","modified_gmt":"2021-07-22T12:16:33","slug":"amp-pages-validation-fail-due-to-custom-changes-in-robots-txt-file","status":"publish","type":"post","link":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting\/robots-blocked","title":{"rendered":"AMP Pages validation fail due to custom changes in &#8220;robots.txt&#8221; file"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In some cases your <a href=\"\/magento-amp\">Magento 2 AMP pages<\/a> will not be validated &#8211; it can be due to custom changes in &#8220;robots.txt&#8221; file.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>1. Enter your website URL with GET parameter &#8220;?amp=1&#8221; into this field and run the validation.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>2. If you receive the following error, this means that an access for Google crawlers is restricted in the &#8220;robots.txt&#8221; file on the server.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1206\" height=\"633\" src=\"\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1.jpeg\" alt=\"Amp search unavailable robots.jpg\" class=\"wp-image-18974\" srcset=\"https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1.jpeg 1206w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1-300x157.jpeg 300w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1-1024x537.jpeg 1024w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1-768x403.jpeg 768w\" sizes=\"auto, (max-width: 1206px) 100vw, 1206px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Solution:<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Remove the restrictions for Google crawlers on your server in order your AMP pages to be parsed.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In some cases your Magento 2 AMP pages will not be validated &#8211; it can be due to custom changes in &#8220;robots.txt&#8221; file. 1. Enter your website URL with GET parameter &#8220;?amp=1&#8221; into this field and run the validation. 2. If you receive the following error, this means that an access for Google crawlers is &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting\/robots-blocked\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;AMP Pages validation fail due to custom changes in &#8220;robots.txt&#8221; file&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[105],"tags":[],"class_list":["post-17669","post","type-post","status-publish","format-standard","hentry","category-troubleshooting","entry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>AMP Pages validation fail due to custom changes in &quot;robots.txt&quot; file - Plumrocket Documentation<\/title>\n<meta name=\"description\" content=\"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in &quot;robots.txt&quot; file. 1. Enter your website URL with GET\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AMP Pages validation fail due to custom changes in &quot;robots.txt&quot; file - Plumrocket Documentation\" \/>\n<meta property=\"og:description\" content=\"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in &quot;robots.txt&quot; file. 1. Enter your website URL with GET\" \/>\n<meta property=\"og:url\" content=\"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked\" \/>\n<meta property=\"og:site_name\" content=\"Plumrocket Documentation\" \/>\n<meta property=\"article:published_time\" content=\"2021-03-09T09:25:43+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-07-22T12:16:33+00:00\" \/>\n<meta name=\"author\" content=\"Plumrocket\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Plumrocket\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AMP Pages validation fail due to custom changes in \"robots.txt\" file - Plumrocket Documentation","description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","og_locale":"en_US","og_type":"article","og_title":"AMP Pages validation fail due to custom changes in \"robots.txt\" file - Plumrocket Documentation","og_description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","og_url":"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","og_site_name":"Plumrocket Documentation","article_published_time":"2021-03-09T09:25:43+00:00","article_modified_time":"2021-07-22T12:16:33+00:00","author":"Plumrocket","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Plumrocket","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#article","isPartOf":{"@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting\/robots-blocked"},"author":{"name":"Plumrocket","@id":"https:\/\/plumrocket.com\/docs\/#\/schema\/person\/c96fccdb89342ae6804272265723eca8"},"headline":"AMP Pages validation fail due to custom changes in &#8220;robots.txt&#8221; file","datePublished":"2021-03-09T09:25:43+00:00","dateModified":"2021-07-22T12:16:33+00:00","mainEntityOfPage":{"@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting\/robots-blocked"},"wordCount":92,"articleSection":["Troubleshooting"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting\/robots-blocked","url":"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","name":"AMP Pages validation fail due to custom changes in \"robots.txt\" file - Plumrocket Documentation","isPartOf":{"@id":"https:\/\/plumrocket.com\/docs\/#website"},"datePublished":"2021-03-09T09:25:43+00:00","dateModified":"2021-07-22T12:16:33+00:00","author":{"@id":"https:\/\/plumrocket.com\/docs\/#\/schema\/person\/c96fccdb89342ae6804272265723eca8"},"description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","breadcrumb":{"@id":"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked"]}]},{"@type":"BreadcrumbList","@id":"\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Magento 2 Extensions","item":"https:\/\/plumrocket.com\/magento-extensions"},{"@type":"ListItem","position":2,"name":"Magento 2 AMP","item":"https:\/\/plumrocket.com\/magento-amp"},{"@type":"ListItem","position":3,"name":"Documentation","item":"https:\/\/plumrocket.com\/docs\/magento-amp"},{"@type":"ListItem","position":4,"name":"v2","item":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2"},{"@type":"ListItem","position":5,"name":"Troubleshooting","item":"https:\/\/plumrocket.com\/docs\/magento-amp\/v2\/troubleshooting"},{"@type":"ListItem","position":6,"name":"AMP Pages validation fail due to custom changes in &#8220;robots.txt&#8221; file"}]},{"@type":"WebSite","@id":"https:\/\/plumrocket.com\/docs\/#website","url":"https:\/\/plumrocket.com\/docs\/","name":"Plumrocket Documentation","description":"Extensions docs, troubleshootings etc.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/plumrocket.com\/docs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/plumrocket.com\/docs\/#\/schema\/person\/c96fccdb89342ae6804272265723eca8","name":"Plumrocket","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/30d7e8cb4567485b56880a1658849a1252ac12806e8f95b3be71e986bf5eabee?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/30d7e8cb4567485b56880a1658849a1252ac12806e8f95b3be71e986bf5eabee?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/30d7e8cb4567485b56880a1658849a1252ac12806e8f95b3be71e986bf5eabee?s=96&d=mm&r=g","caption":"Plumrocket"}}]}},"_links":{"self":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/17669","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/comments?post=17669"}],"version-history":[{"count":1,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/17669\/revisions"}],"predecessor-version":[{"id":22281,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/17669\/revisions\/22281"}],"wp:attachment":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/media?parent=17669"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/categories?post=17669"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/tags?post=17669"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}