Check your website's robots.txt file to verify it exists, is accessible, and doesn't block search engines from crawling important pages.
Want to check the whole page? Try the On-Page SEO Checker.
Checks for the existence of the file at the site root to ensure search engine crawlers have clear navigation guidelines.
Tests the target URL against active rules to instantly determine whether search engines are allowed or disallowed from crawling the page.
Pinpoints the exact line number and directive text in the robots.txt file that triggers a block for faster debugging.
Provides a direct external link to the live robots.txt file for rapid manual inspection.