Tools / Bulk URL status checker
Bulk URL status checker
Paste a list of URLs, one per line, and pick up to three agents to check them as — a browser, a search engine crawler, an AI bot or a link-preview bot. You get each URL’s HTTP status and response time per agent side by side, up to 30 URLs at a time. A scheme is added for you if you leave it off.
Go beyond status codes
A URL that responds can still be insecure or slow. Check these next.
Related free tools
How it works
- Pick up to three agents. Each URL is requested once per agent with that agent’s
User-Agentheader, all in parallel, so a long list still comes back quickly. - Redirects are shown, not followed — a
301/302row displays itsLocationso you can see where it points without the status being masked by the destination. - A
HEADis tried first, with a rangedGETretry for servers that refuseHEAD, so a working URL is not mislabelled as down. - Every status comes with its response time — time to first byte from our server, which covers the connection, TLS handshake and the server’s own processing. Anything over 1.5 s is flagged amber, and the summary gives each agent’s median and slowest time.
- Up to 30 URLs per run. Hosts that do not resolve to a public address are skipped and counted.
Which agents you can check as
- Browsers — Chrome, Firefox, Edge and Opera on desktop, Safari on iPhone. The baseline: what a human visitor gets.
- Search engines — Googlebot (smartphone), Bingbot and DuckDuckGo’s DuckDuckBot. A page that answers browsers but not crawlers will not be indexed.
- AI — GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, DuckDuckGo’s DuckAssistBot and Common Crawl’s CCBot. Shows whether your firewall or CDN lets AI crawlers and assistants read the page.
- Social — the Facebook, X, LinkedIn and Slack link-preview bots. A blocked preview bot means a bare link with no title or image when someone shares the URL.
- Other — plain
curland Pythonrequests, the strings most scripts and uptime monitors send.
With more than one agent, a By agent summary comes first and each row lists every agent’s status, so a URL that serves Chrome a 200 but GPTBot a 403 stands out. The Quick picks fill all three slots for the most common comparisons.
Reading the colours
- Green —
2xx, a normal success. - Amber — a
3xxredirect (with its target shown), a4xxother than 404/410 (often auth or rate-limiting), or a response that took over 1.5 s to start. - Red —
404/410, a5xxserver error, or no response at all.
What it is good for
Confirming a batch of links from a spreadsheet, a newsletter or a sitemap before you publish; spot-checking that a list of pages survived a migration; or re-checking a set of URLs after a DNS or hosting change. It is a status probe, not a crawler — it checks exactly the URLs you give it and does not discover new ones. To extract and check every link on a single page instead, use the broken link checker; to trace one URL’s full redirect chain hop by hop, use the redirect chain tracer.
A note on bot checks
The request carries the agent’s User-Agent string, but it comes from our server, not from Google, OpenAI or Anthropic. Sites that verify crawlers by IP range or reverse DNS may treat it as an impostor and answer differently than they would the real bot. A result here shows how your site reacts to the name; to confirm the real crawler gets through, check your server logs or the search engine’s own URL inspection tool.
A note on 403 and 429
A 403 or 429 from a single automated probe does not always mean the URL is broken — many sites behind a CDN or WAF answer an unfamiliar datacenter client that way while serving browsers normally. Treat those as “check by hand,” not as confirmed failures. Hard 404/410 and 5xx responses are the reliable signals that something is actually wrong.
Frequently asked questions
Why does my page return 200 to Chrome but 403 to GPTBot?
Something in front of the site is blocking that User-Agent: a CDN or WAF bot rule (Cloudflare’s “block AI bots” setting is a common one), a security plugin, or a server rule. If you want AI assistants to read and cite the page, allow the agent there. Blocking it in robots.txt is a separate, polite request that does not produce a 403.
Is a result for Googlebot here the same as Google’s own crawl?
Not always. We send Googlebot’s User-Agent string, but not from Google’s IP addresses. Sites that verify Googlebot by reverse DNS may block or challenge this request while letting the real crawler in. Use Search Console’s URL inspection to confirm what Google itself receives.
Why can I only pick three agents?
Each agent multiplies the requests sent to your server: 30 URLs as 3 agents is 90 requests in one burst. Three covers the usual comparison (a browser, a search crawler and an AI bot) without hammering the site you are checking.
Is the response time the same as page speed?
No. It is time to first byte: how long the server takes to start answering, measured from our server with every request in parallel. A slow first byte usually means slow hosting, no caching or heavy server-side work, and it delays everything else. It does not include images, scripts or rendering — for full load times and Core Web Vitals, use the website speed test.
Does this tool follow redirects?
No. A 301 or 302 is reported with its Location, so you see exactly what each agent was told. Some sites redirect bots and phones to different places, and following the redirect would hide that. To trace a full chain hop by hop, use the redirect checker.
What do “blocked/4xx” and “broken” mean in the By agent summary?
Broken is a hard failure: 404, 410, a 5xx, or no response at all. Blocked/4xx is any other 4xx — usually 401, 403 or 429 — which for a bot agent most often means a firewall or rate limit turned it away rather than that the page is missing.