A sitemap is meant to be the least interesting file a website publishes. It lists pages. Crawlers read it. Nobody looks at it twice.
Open Library publishes one of the largest ones on the public web. The index declares thousands of child files, every one gzipped, every one a contiguous block of catalogue identifiers. We downloaded all of them and counted every URL inside.
The total is enormous and unremarkable. What is remarkable is a stretch in the middle where the file keeps going and the pages stop.
A run of 98 child sitemaps, covering close to a million identifier slots, holds 977 URLs between them. Not 977 each. 977 in total. The files either side of the run are ordinary. The run itself is a hole with sharp edges.
Results at a glance
Open Library's sitemap index points at 6,117 child files listing 55.79 million URLs, split between work pages and author pages. We counted every file, not a sample of them.
Inside that index sits a run of 98 consecutive files that are almost entirely empty, and 3 more buckets that are missing from the index altogether. The site is fine. The map has holes in it, and the holes are the interesting part.
What a dead zone actually is
Open Library gives every record a permanent identifier, and it builds its sitemap by chopping that identifier space into blocks of 10,000. Each child file is one block. A file's size is therefore not a publishing decision at all — it is simply a report of how many identifiers in that range were ever assigned.
Most blocks are close to full. The typical works file lists somewhere near 9,700 URLs, which means the catalogue has grown densely and continuously through almost the whole range.
Then, partway through, it does not. For 98 blocks in a row the fill rate collapses to a tenth of a percent. Each of those files still exists, still downloads, still parses, still declares itself a sitemap. It just has next to nothing in it. The boundaries are abrupt: the file immediately before the run is a normal working file, and the file immediately after picks up again, thinly at first.
None of this breaks anything. A crawler that fetches all 98 files loses a little time and gains almost nothing. But a sitemap is a budget request, and this one asks for attention it does not need.
The map covers only two things
The second finding is quieter and probably matters more. Every single URL in the sitemap is either a work page or an author page. There is no third pattern anywhere in the index.
That leaves real, working, publicly reachable parts of the site outside the map entirely. Edition pages resolve. Subject pages resolve. The homepage resolves. None of them appear.
This is worth separating from the dead zone, because they pull in opposite directions. One is a large stretch of declared pages that barely exist. The other is a large body of existing pages that is never declared. If you were auditing this site cold, you would find the first by accident and miss the second entirely, because nothing in the index hints that anything is absent.
The practical lesson is not that Open Library made an error. It is that a sitemap describes the generator that produced it, not the website. Ours would too. So would yours.
Why the file count runs high
The sitemap specification allows 50,000 URLs per file. Packed to that limit, this catalogue would need somewhere around 1,120 files. Open Library uses 6,117 — about 5.5 times more than necessary.
The reason is the same bucketing decision that produced the dead zone. Fixed identifier blocks are trivially cheap to regenerate: when a record is added, exactly one file changes and everything else can stay untouched. Packing files to the limit would mean rewriting the tail of the index every time the catalogue grows.
That is a defensible trade, made once, and the dead zone is its side effect. You do not get the cheap incremental rebuild without also getting empty buckets wherever the identifier space was never filled.
Links do not scale with pages
We ran the domain through our backlink tool, and the ratio it produced is the lowest we have recorded anywhere in this series. Divide referring domains by pages and Open Library lands at 0.0029 — roughly a ninth of levels.fyi, which held the previous floor at 0.026.
Read that as a fact about the shape of the site rather than a failure. Sites at the top of our table earn links per page by design: calculator.net does it because each of its 222 pages is a destination someone bookmarks and cites. A catalogue of tens of millions of records earns links at the level of the collection, not the record. The average work page will never be cited by anyone, and it does not need to be.
What it does mean is that internal linking carries almost the entire weight of discovery here. External signals cannot reach that far down.
What the audit says about crawling
Our site audit scored openlibrary.org at 73, which sits below Healthline's 74 — the lowest score we had previously recorded in this series. It flagged 1 critical issue and passed 108 checks.
Read the sub-scores rather than the headline and the picture is consistent with everything above. Technical structure is near-clean. Mobile is clean. The weakness is concentrated in speed, and speed is exactly where a site serving millions of database-backed detail pages under a self-imposed crawl delay is expected to give ground.
Open Library's robots.txt asks major crawlers to wait 10 seconds between requests. That is a deliberate choice to protect a public service from the traffic its own size invites. It costs points in an automated audit. It is not obviously the wrong call.
Counting it twice changed the answer
We counted this sitemap twice with two independent passes, and we should be plain about why.
The first pass was a stratified sample: every file's compressed size measured, the small ones counted exactly, the dense ones sampled systematically. The second pass downloaded all 6,117 files and counted every URL. The sample came in 0.311% high, biased upward in both segments — the signature of an estimate that under-weights a sparse tail it never saw enough of.
The number we publish is the full count. The sample was good work and it was still wrong, in a direction that a dead zone of this kind guarantees.
Copy this in an afternoon
You do not need a catalogue of millions to apply any of this. 4 steps, in order.
1. Download your own sitemap index and count it. Not the number your CMS reports — the number you get by fetching every child file and counting URLs. If the two disagree, the disagreement is the finding.
2. Sort your child files by URL count. Anything sitting far below the median is a bucket you are asking crawlers to fetch for nothing. A handful is noise. A consecutive run is a generator bug worth understanding.
3. List every page type your sitemap does not contain. Crawl your own navigation, collect the URL patterns, and subtract the ones the sitemap declares. Whatever is left is a whole class of pages you are not asking anyone to index.
4. Check that lastmod is telling the truth. If every file carries an identical timestamp, the field is decoration. Either make it accurate per file or accept that crawlers will ignore it.
What you cannot copy
The unfair advantage is the identifier space itself. Open Library's pages exist because the records exist, accumulated over a long period into a public catalogue that cannot be bought, scraped into being, or generated on demand. Every structural quirk in this article — the fixed blocks, the empty run, the missing buckets — is downstream of that. You can copy the sitemap architecture in a day. You cannot copy the thing it maps.
And here is what we could not check:
We counted the sitemap, not the index. Every figure here describes what Open Library declares. We have no visibility into how many of those URLs any search engine actually crawled, indexed, or ranked, and no traffic data of any kind.
Our first count was an estimate and it was off. The sampled pass returned 55,962,947 against the census figure of 55,789,489. We publish the census. 2 figures the sampled pass derived — an average works-file size and a density ratio built on it — did not reproduce under the full count, and we have dropped them rather than round them.
Our backlink tool does not reconcile with itself. Its dofollow and nofollow figures add up to about 5% more than the total it reports on the same screen. That is our defect, not Open Library's. We publish both numbers in the FAQ so you can see the gap rather than take a cleaned-up version of it.
We cannot say why the dead zone is there. We know exactly where it starts and stops and how empty it is. Whether it reflects a data migration, a reserved identifier range, or something else entirely is not something a crawler can determine, and we are not going to guess.
Freshness is unreadable. Every child file carries the identical lastmod timestamp, so nothing in the sitemap tells us which parts of the catalogue changed recently. Our snapshot is one day in August 2026 and all of it will drift.
Dead sitemap buckets, missing page types and crawl-budget waste do not show up in analytics. They show up in a crawl. Run yours and see what your own map is actually declaring.
- Technical, on-page, speed and mobile scored separately
- Critical issues, warnings and notices ranked by impact
- Full report in your browser, nothing to install
Frequently asked questions
How many URLs does Open Library's sitemap actually list?
55,789,489, from a full census in which all 6,117 child sitemaps were downloaded and every URL counted. That splits into 40,718,247 work pages across 4,496 files and 15,071,242 author pages across 1,621 files.
What exactly is the dead zone?
A run of 98 consecutive child sitemaps covering identifier blocks OL13860000W to OL14839999W — 980,000 slots. Between them they hold 977 URLs, a fill rate of 0.0997%. Against a normal works file's average of roughly 9,258 URLs, that is a density gap of about 930 to 1.
Are those the only gaps in the index?
No. 3 further buckets are absent from the index entirely and return 404 on the server, accounting for another 30,000 identifier slots. Counting a few other sparse patches, the full sub-100-URL footprint is 102 files holding 1,130 URLs between them.
Does the sitemap cover the whole site?
No, and this is the larger omission. Every URL in it is a work page or an author page. Edition pages and subject pages both return 200 to a browser, as does the homepage, and none of them appear anywhere in the index.
Why are there 6,117 files when far fewer would do?
The sitemap spec caps a file at 50,000 URLs, so this catalogue would fit into roughly 1,120 files if packed. Open Library instead buckets by fixed 10,000-identifier blocks, which uses about 5.5 times more files but makes incremental regeneration cheap.
What did the backlink tool report?
A domain rank of 91, level with Rotten Tomatoes, and 159,686 referring domains. It reported 65,397,712 total backlinks alongside a split of 60,623,751 dofollow and 8,034,333 nofollow, which is 88.3% dofollow. Note that the split sums to about 5% more than the stated total — an inconsistency in our tool.
How does 0.0029 compare with the rest of the series?
It is the lowest we have measured. Levels.fyi held the previous floor at 0.026 across 528,363 pages, and Meetup sat at 0.069. At the other end, calculator.net earns 143 referring domains per page across just 222 pages.
What did the site audit find?
A score of 73 with 1 critical issue, 5 warnings, 6 notices and 108 checks passed. By section: on-page 21/25, technical 63/67, speed 23/29 and mobile 2/2. Open Library also sets a 10-second crawl delay for major crawlers in robots.txt.
The temptation with a site this size is to treat the sitemap as a rounding error — 55.79 million URLs, a few thousand of them empty, who cares. That reading misses what the shape is telling you. The empty run and the missing page types come from the same place: a generator that describes an identifier space rather than a website.
Most sites have a smaller version of the same problem, and it is invisible for the same reason. The sitemap is generated, it validates, it submits cleanly, and nobody ever opens it. The only way to find out what yours actually declares is to fetch it and count.
That is a half-hour job on a normal site, and the UNmiss Website Audit will do the crawling half of it for you, for free, in your browser.
Measured on 29 August 2026. Page counts come from a full census of Open Library's sitemap index: all 6,117 child sitemaps downloaded with a browser user agent and every URL counted, with zero fetch failures. Backlink and audit figures are readings from UNmiss Backlink Analyzer and UNmiss Website Audit on that date.
Third-party metrics are one tool's view on one day and will drift. Open Library is not affiliated with UNmiss, was not contacted for this analysis and did not take part in it.