Broken-link audits: finding every 404 before your users do

Why a sitemap-based check misses half the problems, why a soft 404 is worse than a real one, and how to run a recurring audit.

Links · Published: 5 August 2026 · 3 min read · Дмитрий Орлов

A broken link rarely arrives alone. It is usually a symptom: a renamed section, a redesign, a decommissioned subdomain — and dozens of dead addresses spread across the site that nobody knows about until a customer writes in.

Why sitemap.xml alone is not enough

A sitemap-based check only finds the pages you declared yourself. It knows nothing about:

• links in article bodies to external resources that have since closed;
• buttons whose href is assembled in JavaScript;
• page resources — images, fonts, scripts — returning 404;
• links in the footer and menu pointing at pages excluded from the sitemap.

A complete audit is a graph crawl: take a start URL, extract every <a href>, <img src>, <link> and <script src>, recurse within the domain, and record the status code for every address.

Soft 404 — the nastiest case

The server answers 200 OK while the page says "Product not found". For a user it is a dead end; for a search engine it is a valid page, dutifully indexed. You detect it through indirect signals: suspiciously little content, boilerplate phrasing, a missing H1, a redirect to the homepage that loses the path.

What to do with each class of error

• 404 — the page does not exist. Redirect (301) to the closest relevant address, or serve a genuinely useful error page.
• 410 — deliberately removed. Leave it and strip the inbound links.
• 500 — an application error. File a bug, not a redirect.
• 403 — locked by permissions. Check whether it was meant to be public.
• Timeout — a slow external resource. Take it off the critical path or cache it.

Redirect chains are their own category. Three consecutive 301s work, but they cost up to half a second on a slow network and dilute link equity. Collapse chains to a single hop.

Regularity beats completeness

A one-off audit of 10,000 pages produces a handsome report that is stale within a month. More practical:

1. A full crawl once a week, on a schedule.
2. A quick crawl of the key templates after every release.
3. An automated check of external links once a month — they die more often than internal ones.
4. A CI gate: the release fails if new internal 404s appear.

How not to breed redirects

When you move a section, prepare the old-to-new address map before the deploy, not after. One 301 layer in nginx or an edge function is cheaper than ten rules accumulated over the years. And delete old rules once traffic through them drops to zero — otherwise in two years nobody will dare touch that file.