Tools / Bulk URL status checker

Bulk URL status checker

Paste a list of URLs, one per line, and pick up to three agents to check them as — a browser, a search engine crawler, an AI bot or a link-preview bot. You get each URL’s HTTP status and response time per agent side by side, up to 30 URLs at a time. A scheme is added for you if you leave it off.

Check as compare up to 3 agents
Quick picks:

How it works

Which agents you can check as

With more than one agent, a By agent summary comes first and each row lists every agent’s status, so a URL that serves Chrome a 200 but GPTBot a 403 stands out. The Quick picks fill all three slots for the most common comparisons.

Reading the colours

What it is good for

Confirming a batch of links from a spreadsheet, a newsletter or a sitemap before you publish; spot-checking that a list of pages survived a migration; or re-checking a set of URLs after a DNS or hosting change. It is a status probe, not a crawler — it checks exactly the URLs you give it and does not discover new ones. To extract and check every link on a single page instead, use the broken link checker; to trace one URL’s full redirect chain hop by hop, use the redirect chain tracer.

A note on bot checks

The request carries the agent’s User-Agent string, but it comes from our server, not from Google, OpenAI or Anthropic. Sites that verify crawlers by IP range or reverse DNS may treat it as an impostor and answer differently than they would the real bot. A result here shows how your site reacts to the name; to confirm the real crawler gets through, check your server logs or the search engine’s own URL inspection tool.

A note on 403 and 429

A 403 or 429 from a single automated probe does not always mean the URL is broken — many sites behind a CDN or WAF answer an unfamiliar datacenter client that way while serving browsers normally. Treat those as “check by hand,” not as confirmed failures. Hard 404/410 and 5xx responses are the reliable signals that something is actually wrong.

Frequently asked questions

Why does my page return 200 to Chrome but 403 to GPTBot?

Something in front of the site is blocking that User-Agent: a CDN or WAF bot rule (Cloudflare’s “block AI bots” setting is a common one), a security plugin, or a server rule. If you want AI assistants to read and cite the page, allow the agent there. Blocking it in robots.txt is a separate, polite request that does not produce a 403.

Is a result for Googlebot here the same as Google’s own crawl?

Not always. We send Googlebot’s User-Agent string, but not from Google’s IP addresses. Sites that verify Googlebot by reverse DNS may block or challenge this request while letting the real crawler in. Use Search Console’s URL inspection to confirm what Google itself receives.

Why can I only pick three agents?

Each agent multiplies the requests sent to your server: 30 URLs as 3 agents is 90 requests in one burst. Three covers the usual comparison (a browser, a search crawler and an AI bot) without hammering the site you are checking.

Is the response time the same as page speed?

No. It is time to first byte: how long the server takes to start answering, measured from our server with every request in parallel. A slow first byte usually means slow hosting, no caching or heavy server-side work, and it delays everything else. It does not include images, scripts or rendering — for full load times and Core Web Vitals, use the website speed test.

Does this tool follow redirects?

No. A 301 or 302 is reported with its Location, so you see exactly what each agent was told. Some sites redirect bots and phones to different places, and following the redirect would hide that. To trace a full chain hop by hop, use the redirect checker.

What do “blocked/4xx” and “broken” mean in the By agent summary?

Broken is a hard failure: 404, 410, a 5xx, or no response at all. Blocked/4xx is any other 4xx — usually 401, 403 or 429 — which for a bot agent most often means a firewall or rate limit turned it away rather than that the page is missing.