Bulk Indexability Checker (noindex, X-Robots-Tag, canonical)
A page can look perfectly normal in the browser and still be invisible to search engines — a single noindex tag left after a site moved out of staging, an X-Robots-Tag header set by the server config, or a canonical…
More about this tool
pointing at another URL quietly tells crawlers to skip the page. This bulk checker inspects every URL in your list the way a crawler does: it reports the meta robots tag with noindex and nofollow flags, the X-Robots-Tag response header (which works even when the HTML has no meta tag at all), the canonical link with a match check against the final URL after redirects, and the list of hreflang alternates with a per-page count. Anything that closes the page from indexing is highlighted, so auditing a whole section after a redesign or a migration takes minutes: paste the URLs, scan the flagged rows, fix the templates, re-run. The report exports to CSV for documentation and for re-checks after the fixes.
How it works
- Paste up to 100 items, one per line. Duplicates are removed automatically.
- URLs without a scheme are checked over https://.
- The check runs in the background; the results page shows live progress.
- Filter the report by status code and export it to CSV.
Rate limit: 5 jobs per hour from one IP address. Results are stored for 7 days.
FAQ
Meta robots or X-Robots-Tag — what is the difference?
Both carry the same directives, but the meta tag lives in the HTML while X-Robots-Tag is an HTTP response header. The header applies to any resource — PDFs, images — and is usually set in the server or CMS config, which is why pages sometimes carry it with no visible meta tag. This tool checks both.
Does nofollow on a page block indexing?
Not the page itself: a nofollow in the page-wide robots tag asks crawlers not to follow its outgoing links. The page can still be indexed and shown. noindex is the directive that removes the page from search — the report flags both separately.
When is a canonical "not ok"?
When the canonical link points somewhere other than the final URL of the page itself (after redirects). That is sometimes intentional — duplicate parameters, syndicated copies — but on pages meant to rank on their own it usually means a template bug, so the report marks the mismatch for review.
Does a redirect affect the result?
Yes — the tags are read from the final page after following redirects, exactly like a crawler does. The final URL is shown in the report so you always know which page was actually inspected.
Can I check non-HTML files?
Yes. X-Robots-Tag headers apply to any file type and are reported; meta tags and canonical are only read when the response looks like HTML, so for a PDF you get the header-based verdict.
Other tools
Bulk Domain Blacklist Checker (DNSBL)
Free bulk blacklist checker: paste domains and check them against Spamhaus, Barracuda, SpamCop, SORBS and SURBL in one run. Instantly see if your domain or IP is blacklisted. CSV export.
Accepts a list of domainsBulk DNS Records Lookup (A, AAAA, MX, NS, TXT)
Free bulk DNS lookup with a custom DNS server option: paste domains and get A, AAAA, CNAME, MX, NS, TXT, SOA and SRV records grouped by type, one record per line with TTL, MX priorities, copy buttons and CSV export.
Accepts a list of domainsBulk Domain Expiration Checker (RDAP)
Free bulk domain expiry checker via RDAP with automatic WHOIS fallback: paste a list of domains and get the registrar, registration and expiration dates and days left for each. Domains expiring within 30 days are highlighted. Registry answers are cached for 24 hours. CSV export included.
Accepts a list of domainsBulk URL Status Checker
Free bulk URL status checker: paste up to 100 URLs and get HTTP status codes, redirect chains, final URLs and response times. Export the results to CSV.
Accepts a list of URLsBulk Mail Records Checker (SPF, DKIM, DMARC, MX)
Free bulk mail records checker: paste domains and audit SPF, DKIM, DMARC policies and MX records. Make sure your domains can send mail and cannot be spoofed. CSV export.
Accepts a list of domainsBulk Meta Tags Checker (Title, Description, H1)
Free bulk meta tags checker: paste URLs and get Title, meta description, H1 and heading counts with length checks and duplicate detection. Export the SEO audit to CSV.
Accepts a list of URLsBulk Open Graph & Twitter Cards Checker
Free bulk Open Graph checker: paste URLs and audit og:title, og:description, og:image, twitter:card and more social preview tags. Export the report to CSV.
Accepts a list of URLsBulk Outgoing Links & Broken Links Checker
Free bulk outgoing links checker: paste URLs to count external and internal links, dofollow vs nofollow, and find broken links (404, 5xx, dead domains). Export the audit to CSV.
Accepts a list of URLsBulk robots.txt Checker
Free bulk robots.txt checker: paste a list of domains and see the status of /robots.txt, redirect count, file size, the number of Sitemap directives and whether all crawlers are blocked. CSV export included.
Accepts a list of domainsBulk Security Headers Checker (HSTS, CSP)
Free bulk security headers checker: paste domains and audit HSTS, X-Content-Type-Options, CSP, X-Frame-Options, Referrer-Policy, Permissions-Policy and the HTTPS redirect. CSV export.
Accepts a list of domainsBulk SSL Certificate Checker
Free bulk SSL certificate checker: paste a list of domains and instantly see the certificate subject, issuer, hostname match, expiry date and days left before renewal. Find expiring certificates before they become incidents. CSV export included.
Accepts a list of domainsBulk TTFB & Speed Checker
Free bulk TTFB checker: paste a list of URLs and measure time to first byte, total response time and HTML size for each page. Export the report to CSV.
Accepts a list of URLs