Last Updated on June 6, 2025 by Admin
The "Indexed, though blocked by robots.txt" error in Google Search Console means search engines displays the webpages which are blocked by disallow instruction in robots.txt file. It will not index the url again, and will be removed after crawling which results low performance of your website in serp.
What does indexing have to do with robots.txt?
During their crawling of your website, search engine robots, such as Googlebot, look for a file named robots.txt. In this file, the bots read instructions that include URLs or directories that they can or cannot visit and carry out an indexing process. Now, this is completely different from Discovered Currently, Not Indexed error.
When a page is blocked in the robots.txt file, and there are links from other websites, search engines may still decide to index it without considering its content causing "Indexed, though blocked by robots.txt" in Google Search Console.
When a page is disallowed in the robots.txt file, but some external links still point to that page, the crawlers often crawl the disallowed page. Google finds the page through these links and uses link context and relevance as the basis for indexing the page, though it is still struggling to index the page because of unreadable content by its robots.
Different Ways to Fix "Indexed, though blocked by robots.txt"
The effective ways to solve this problem and have your pages indexed by the search engines are as follows:
- Edit robots.txt Directly: The most straightforward solution to this "Indexed, though blocked by robots.txt" issue is to modify your robots.txt file directly. Modify or remove the disallow rule to allow the crawler to access the page or directory that was previously blocked. Removing the “disallow” directive allows search bots to crawl and index the page without problems.
- Use an SEO Plugin: Using a content management system (CMS) like WordPress, you can use SEO plugins to simplify editing your robots.txt file. Main plugins like Yoast SEO and All in One SEO Pack have user-friendly interfaces for modifying the robots.txt file without dealing with the raw code. Tools like RankMath offers users with an easy solution to modify their robot.txt file.
Conclusion
Getting over the "Indexed, though blocked by robots.txt" error may look intricate at first sight. However, once you have all the necessary instruments and knowledge, you can easily deal with them and unlock your website for visitors. If you found this guide helpful, check out more SEO tips and strategies on the AlgoSaga digital marketing agency portal. Online success is possible when you work with a team of professionals committed to making your website rank equally well with competitors.