How to Audit a Website's Metadata at Scale (Without Checking Every Page by Hand)

Metadata problems on a small site are easy to catch — you open the five pages that exist, check the source, done. On a site with a few thousand blog posts, docs pages, or product listings, that approach doesn't scale, and the pages most likely to be broken are exactly the ones nobody thinks to open by hand: the one blog post from three years ago, the product variant page, the docs page that got moved during a restructure. A site-wide metadata problem is rarely a single bad decision. It's usually a template change, a new content type, or a migration that quietly stopped inheriting the defaults everything else gets — and it breaks silently, because a missing og:image doesn't throw an error. It just means the page looks broken when someone shares it.

Why it breaks silently

Metadata gaps tend to come from a small set of recurring causes. A CMS template update changes how the default title or description is generated, and pages created after the change don't get backfilled. A new content type — a landing page builder, a resource library, a job listings section — gets added to the site without anyone wiring it into the metadata layer the rest of the site uses, so every page of that type ships with a generic or blank title. A migration between platforms drops canonical tags or og:image references that pointed at an old asset host. In every case, the page still loads fine in a browser, search results still show something, and the problem only becomes visible when someone shares the link and the preview is blank, wrong, or shows a stock placeholder image. By then it's been live for months.

What 'good' looks like

Before auditing anything, it helps to have a concrete definition of a passing page rather than a vague sense of 'metadata should be there.' The checklist is short and mechanical, which is exactly why it's suited to automation rather than a person eyeballing source code:

• Title tag present and a reasonable length — not empty, not truncated mid-word in search results, not identical to five hundred other pages
• Meta description present and unique to the page, not a copy-pasted boilerplate line reused site-wide
• og:image present and resolving to an actual image — not a 404, not a redirect chain, not an image so large it fails to fetch in time
• Canonical tag present and pointing at the correct URL — not a self-referencing mistake on a paginated or parameterized page, not still pointing at a staging domain
• twitter:card present so link previews degrade correctly on X instead of falling back to a bare link

Spot-checking vs. crawling the whole site

Manual spot-checking has a real place — it's fast, it's free, and it's the right tool when you've just shipped a template change and want to confirm the fix worked on the handful of pages you touched. Its limit is coverage: spot-checking only tells you about the pages you thought to check. It can't tell you about the page you forgot existed, the one generated by a content type nobody remembers is live, or the batch of listings created by an import script eighteen months ago. On a large site, the pages causing the most damage are usually the ones outside anyone's mental model of 'the site,' which is precisely what spot-checking can't find.

A full crawl solves that by treating the sitemap (or the site's actual link graph) as the source of truth instead of a person's memory of what pages exist. It visits every reachable URL, checks it against the criteria above, and comes back with a list instead of a hope. This is the shape of a site-wide audit like the one built into useopengraph: it respects robots.txt so it doesn't touch pages you've deliberately excluded, and it scales from a few dozen pages up to several thousand depending on plan, so the same workflow works whether you're checking a small marketing site or an e-commerce catalog with a large product count. The output is a page-by-page breakdown of what's missing, not a general sense that 'metadata is mostly fine.'

What to do with the findings

Once you have a list of broken or missing metadata, the instinct to work through it alphabetically or in crawl order is understandable but usually wrong. Fix by impact instead: cross-reference the findings against your analytics and start with the pages that actually get traffic or get shared. A missing og:image on your top ten landing pages costs you real impressions and click-through every time someone links to them; the same issue on a page nobody visits costs nothing today. Triaging by traffic also tends to surface the real root cause faster, because your highest-traffic pages are usually your oldest and most-templated ones — fixing the template fixes dozens of pages at once, rather than patching one URL at a time and leaving the underlying cause in place for the next new page to inherit.