{"id":18678,"date":"2021-06-15T14:26:56","date_gmt":"2021-06-15T11:26:56","guid":{"rendered":"https:\/\/plumrocket.com\/docs\/?p=18678"},"modified":"2021-07-22T08:22:34","modified_gmt":"2021-07-22T12:22:34","slug":"amp-pages-validation-fail-due-to-custom-changes-in-robots-txt-file-2","status":"publish","type":"post","link":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","title":{"rendered":"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file"},"content":{"rendered":"\n<p>In some cases your <a href=\"\/magento-amp\">Magento 2 AMP pages<\/a> will not be validated &#8211; it can be due to custom changes in &#8220;robots.txt&#8221; file.<\/p>\n\n\n\n<p><strong>1. Enter your website URL with GET parameter &#8220;?amp=1&#8221; into this field and run the validation.<\/strong><\/p>\n\n\n\n<p><strong>2. If you receive the following error, this means that an access for Google crawlers is restricted in the &#8220;robots.txt&#8221; file on the server.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" width=\"1206\" height=\"633\" src=\"\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg\" alt=\"Amp search unavailable robots.jpg\" class=\"wp-image-18972\" srcset=\"https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg 1206w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-300x157.jpeg 300w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-1024x537.jpeg 1024w, https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots-768x403.jpeg 768w\" sizes=\"(max-width: 1206px) 100vw, 1206px\" \/><\/figure>\n\n\n\n<h2>Solution:<\/h2>\n\n\n\n<p>Remove the restrictions for Google crawlers on your server in order your AMP pages to be parsed.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In some cases your Magento 2 AMP pages will not be validated &#8211; it can be due to custom changes in &#8220;robots.txt&#8221; file. 1. Enter your website URL with GET parameter &#8220;?amp=1&#8221; into this field and run the validation. 2. If you receive the following error, this means that an access for Google crawlers is &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[106],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v17.2 (Yoast SEO v17.2) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file - Plumrocket Documentation<\/title>\n<meta name=\"description\" content=\"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in &quot;robots.txt&quot; file. 1. Enter your website URL with GET\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file\" \/>\n<meta property=\"og:description\" content=\"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in &quot;robots.txt&quot; file. 1. Enter your website URL with GET\" \/>\n<meta property=\"og:url\" content=\"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked\" \/>\n<meta property=\"og:site_name\" content=\"Plumrocket Documentation\" \/>\n<meta property=\"article:published_time\" content=\"2021-06-15T11:26:56+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-07-22T12:22:34+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Plumrocket\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file - Plumrocket Documentation","description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","og_locale":"en_US","og_type":"article","og_title":"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file","og_description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","og_url":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","og_site_name":"Plumrocket Documentation","article_published_time":"2021-06-15T11:26:56+00:00","article_modified_time":"2021-07-22T12:22:34+00:00","og_image":[{"url":"https:\/\/plumrocket.com\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Written by":"Plumrocket","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/plumrocket.com\/docs\/#website","url":"https:\/\/plumrocket.com\/docs\/","name":"Plumrocket Documentation","description":"Extensions docs, troubleshootings etc.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/plumrocket.com\/docs\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#primaryimage","inLanguage":"en-US","url":"\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg","contentUrl":"\/docs\/wp-content\/uploads\/2021\/06\/Amp_search_unavailable_robots.jpeg"},{"@type":"WebPage","@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#webpage","url":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked","name":"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file - Plumrocket Documentation","isPartOf":{"@id":"https:\/\/plumrocket.com\/docs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#primaryimage"},"datePublished":"2021-06-15T11:26:56+00:00","dateModified":"2021-07-22T12:22:34+00:00","author":{"@id":"https:\/\/plumrocket.com\/docs\/#\/schema\/person\/38b360639b934d6c984ee4f3ffce7d20"},"description":"In some cases your Magento 2 AMP pages will not be validated - it can be due to custom changes in \"robots.txt\" file. 1. Enter your website URL with GET","breadcrumb":{"@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting\/robots-blocked#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Magento 2 Extensions","item":"https:\/\/plumrocket.com\/magento-extensions"},{"@type":"ListItem","position":2,"name":"Magento 2 AMP","item":"https:\/\/plumrocket.com\/magento-amp"},{"@type":"ListItem","position":3,"name":"Documentation","item":"https:\/\/plumrocket.com\/docs\/magento-amp"},{"@type":"ListItem","position":4,"name":"v3","item":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3"},{"@type":"ListItem","position":5,"name":"Troubleshooting","item":"https:\/\/plumrocket.com\/docs\/magento-amp\/v3\/troubleshooting"},{"@type":"ListItem","position":6,"name":"AMP Pages validation fail due to custom changes in \u201crobots.txt\u201d file"}]},{"@type":"Person","@id":"https:\/\/plumrocket.com\/docs\/#\/schema\/person\/38b360639b934d6c984ee4f3ffce7d20","name":"Plumrocket","image":{"@type":"ImageObject","@id":"https:\/\/plumrocket.com\/docs\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/72392a92ae750c66560be284502b6676?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/72392a92ae750c66560be284502b6676?s=96&d=mm&r=g","caption":"Plumrocket"},"sameAs":["https:\/\/plumrocket.com\/docs"]}]}},"_links":{"self":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/18678"}],"collection":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/comments?post=18678"}],"version-history":[{"count":1,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/18678\/revisions"}],"predecessor-version":[{"id":22290,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/posts\/18678\/revisions\/22290"}],"wp:attachment":[{"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/media?parent=18678"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/categories?post=18678"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/plumrocket.com\/docs\/wp-json\/wp\/v2\/tags?post=18678"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}