Building Bibliographic Superwork Clusters for Discovery with Local LLMs
Judging work relationships in 88K clusters
Aug 13 2026
In bibliographic description you are able to build relationships to external authorities and records in various ways. In MARC you can use controlled vocabularies which eventually evolved the ability to use identifiers and URIs to establish connections. You can link to other titles specifically via a Name Title authority or some description in the higher 7XX fields mostly used for Serial cataloging. As we move into more FRBR-LRM/WEMI/Linked Data aligned standards like Bibframe relationships become more central to how the record is constructed. The Work and the Instance are separate resources, the Bibframe Hub takes the place of and expands the functionality of Name Title Authorities for cataloging purposes. There are more details Iβm glossing over here because Iβm thinking about the type of relationships that can be built for discovery rather than cataloging. The following little experiment is looking at automated ways to build relationships that aggregate works under the idea of a Superwork: The novel, the translation, the movie adaptation, the 13 different editions, all linked with their relationships defined.
While Hubs do collocation they are similar to Named Title authorities in that they are scoped to specific levels of expression and like all authority work it takes additional effort while cataloging meaning there are very specific rules to when one is created. This idea of Superwork is much more expansive and is of course not a new idea, lots of debate about the role of Works vs something higher. The Share-VDE project implemented a Superwork level they called an Opus that is discussed in this paper. So it’s not a novel idea, it might be a good idea, but the problem is that it’s hard to implement for a number of reasons.
Clustering bibliographic data is also a well trodden path. Twenty-five years ago orgs like OCLC were clustering works for FRBR-ization efforts. The process involves matching parts of MARC records on various fields to create groupings of resources. So clustering is “easy” but once they are clustered how do you relate them to one another? You have to come up with elaborate rule based processes that quickly fall apart against records described at different levels of completeness, conflicting historical practices and an infinite long tail of problems.
Enter the idea of using a LLM to help you judge how all these works are related to one another. I’m an AI pragmatic, I’ve been subjected to a couple demos and pitches this year of AI “cataloging” books and I find that sort of approach to AI in libraries really harmful and gross. I think LLMs can potentially be useful for specific tasks or workflow automations. And I think it has a place in this idea of enriching legacy data. I’ve also been really interested in using local models that can run on your own (beefy) computer as alternatives to expensive black box frontier models.
So this test is going to be:
For the source data I’m going to use the Library of Congress catalog. I do work there but this is a side project (π« ) and I’m using their public downloadable bulk data. To cluster this catalog I wanted to cast a ridiculously wide net, so it looks for things that have the same title and a matching 1XX or 7XX contributor. This will miss things like translated titles (Der Idiot vs The Idiot) or transliterated titles but I use Name Title authorities to try to fill in those gaps. Applying this insane clustering methodology we get the following breakdown out of the ~20M records:
| Cluster size | Clusters | % of clusters | Resources | % of resources |
|---|---|---|---|---|
| 2 | 1,068,038 | 75.9% | 2,136,076 | 51.8% |
| 3β4 | 245,632 | 17.5% | 802,334 | 19.5% |
| 5β9 | 69,157 | 4.9% | 424,928 | 10.3% |
| 10β19 | 15,853 | 1.1% | 204,028 | 4.9% |
| 20β40 | 4,846 | 0.3% | 130,465 | 3.2% |
| 41β100 | 2,007 | 0.14% | 123,284 | 3.0% |
| 101β500 | 905 | 0.06% | 157,027 | 3.8% |
| 501β1,000 | 57 | 0.004% | 38,636 | 0.9% |
| 1,001+ (max 21,188 β generic-title noise) | 30 | 0.002% | 107,475 | 2.6% |
| Total | 1,406,525 | 100% | 4,124,253 | 100% |
So only 20% of records possibly belong to a cluster. The biggest cluster is 2 things, these clusters are often artifacts of MARC cataloging where two formats of the same work are recorded separately (print + ebook for example). For my test I wanted to do a significantly challenging cluster size, but don’t want to do everything. I picked the 5-40 slice as a good test group. This is how that grouping breaks down:
| Cluster size | Clusters | % of clusters | Resources | % of resources |
|---|---|---|---|---|
| 5β9 | 67,821 | 76.6% | 416,411 | 54.8% |
| 10β19 | 15,657 | 17.7% | 201,737 | 26.6% |
| 20β40 | 4,910 | 5.5% | 132,456 | 17.4% |
| 41β100 | 138 | 0.16% | 7,569 | 1.0% |
| 101β340 | 8 | 0.01% | 1,248 | 0.16% |
| Total | 88,534 | 100% | 759,421 | 100% |
While the cutoff was 40 records Name Title authority merging collapsed 89,856 β 88,534 clusters (β1,322) with no change in resources. 146 merged clusters now exceed the original 40-record judging cap.
So we have 88K clusters of varying sizes to judge. We now need a vocabulary to organize how these things relate to each other. I chose to use the local LLM to help build the relationship properties based on data present in the cluster to make a limited vocabulary. There are vocabularies out there for work relationships, which might have been sufficient but I wanted to try a bottom up approach. Though after it was created I did want to compare to what already exists, specifically the Bibframe work to expression relationship vocabulary and the RDA Registry Work relationships. Here is the list of the organic relationship vocabulary and how it maps to the two established ones:
As I mentioned earlier we are going to use a local LLM to judge these clusters of works, basically prompting with the titles of each cluster and metadata data along with the possible relationship list. I’m using Qwen3.6-35B-A3B-bf16 model which is a 35 billion parameter model but only ever has 3 billion parameters active. This is a Mixture-of-Experts (MoE) model that allows it to draw on a lot of parameters but is able to execute quickly. Running 4-6 prompts in parallel results in ~17 tokens a second each on my M5 128GB machine. Even though that is a decent clip it still ran in the background for a couple weeks to get through all 88K clusters. If you ran a job like this using the same model on a provider like Open Router it would cost around $150-200 in tokens but probably take a day rather than weeks. If you ran it on something like Gemini it would cost MUCH more but probably better quality. So it’s all a trade off, money vs time vs quality vs local, etc.
Now that everything is judged I wanted to first evaluate how well the LLM worked and then break down how the clusters panned out. To judge them I did two approaches, I manually reviewed 200 clusters and judged their accuracy and I also had a frontier model (Opus 4.8) review 2500 and judge. There are two factors when thinking about correctness here, there are 4 broad buckets:
Each of the relationships roll up into one of these buckets. So a title in the cluster could be generally correct but specifically incorrect. The results:
| How strictly we grade | Overall: judge matched the grader accuracy (micro) | When it picks a label, the label is rightΒΉ macro-P | It finds all the records a label applies toΒΉ macro-R | Balanced score: every label counts equally macro-F1 | Balanced score: common labels count more weighted-F1 | |
|---|---|---|---|---|---|---|
| Frontier model (Opus) 2,500 records | Exact property required (45 choices) | 76.3% | 80.4% | 75.2% | 72.8% | 74.8% |
| Right broad category (4 buckets) | 91.0% | 81.6% | 60.2% | 65.7% | 90.0% | |
| Human (Matt) 200 records | Exact property required (19 labels seen) | 88.6% | 91.1% | 90.8% | 87.7% | 88.8% |
| Right broad category (4 buckets) | 96.6% | 97.3% | 89.9% | 93.0% | 96.4% |
ΒΉ averaged over labels, so rare labels weigh as much as common ones
I was more generous while grading than the model, out of +40 possible properties the model tends to be very specific, a useful stat that indicates quality is probably the 4 broad category score, I gave it a 96% the model a 91%. Below are more views into the grading results:
Each record gets one label from a 44-term controlled vocabulary. A handful dominate: most records are simply another edition, printing, or part of the same work.
Collapsing the 44 labels into four buckets. The clustering's core job β deciding membership β is overwhelmingly "yes, this belongs," with a small, important slice excluded as coincidental.
Records by material type, and how big the clusters get. Most clusters are small; a long tail of generic titles runs to dozens of records.
Cluster size distribution (records per cluster)
Two independent graders scored a stratified sample: a human expert (you) on a population-representative set, and a frontier model (Opus) on 2,500 records. The gap between exact-label and bucket-level accuracy is the story β most "errors" are near-ties inside the right bucket.
At the bucket level both graders agree the model is right ~90β97% of the time. Opus is the stricter grader on exact labels.
Frontier-graded accuracy per material (population-weighted; darker = stronger). Books, maps, video and serials are strong; images and software are where the model struggles.
Every relationship label graded at least 15 times, as a pair of dots: precision (dark β when the model uses the label, how often it's right) and recall (light β how often the true cases actually get it). Sorted by the gap between the two.
One machine, two failure modes. Specific labels (illustrated_edition, related_work) are trustworthy when they appear but rarely used β the record usually gets a generic label instead. And the generic favorites (later_printing_reprint, format_variant, source_work) are what absorb them: they catch nearly everything, wrongly a third to half of the time. Hover a row for exact values and sample size.
Only one label removes a record from a cluster: different_work. Here's precision (when it excludes, is it right?) and recall (does it catch the true intruders?) β human vs Opus.
Both agree the model over-excludes ~1 in 5 (precision ~80%). Opus's lower recall was Opus itself being over-strict: on 15/15 boundary cases where Opus wanted a record thrown out, the human sided with the model.
On the records both reviewed. Agreement with the model, and between the judges (Cohen's ΞΊ = 0.48, "moderate").
1.5% of clusters (1,374) had most records excluded β a generic title swept up unrelated works. But most "failures" aren't junk:
74% of failed clusters are actually several real works merged under one title, not garbage β the judge just can't re-partition them.
Example β "the gift": 21 records = 9 different novels by 9 authors (Larriva, Garwood, Jensen, Ashley, Stover, β¦) that share only the title. The right answer is ~9 tiny clusters; the pipeline kept 1 and excluded the rest.
The results are pretty straight forward: It works mostly, needs refinement, models will improve or more capable could be used, more metadata pre/post-processing would also help things. But I think the context of its use is also important. This sort of clustering would be enriching a discovery layer, detached from record metadata. Which I think is a good approach to anything AI, progressive enhancement, if it works that’s great and makes the user experience better if it fails, oh well doesn’t really prevent the user from completing their use case.
I made a very OPAC-ish search and display interface for the 88K clusters: https://thisismattmiller.github.io/superwork/
The best way to try it out is to just click the RANDOM button a few times and see what comes up. But here are a few links:
There is also a more analytical QA type of view that you can facet on material and LCC here.
Notes about this first attempt:
My takeaway from this test: Yes it is possible to use a “low” power local LLM to do bibliographic work like this, but quality probably scales with model size. Yes it seems potentially useful for discovery purposes, but there would need to be a lot more data work done to get it closer to production quality. Overall I think there is a place for the types of processes that exist to evaluate data that is too complex to be organized via simple rules and could be useful for enriching legacy catalog data.
Code + a lot of markdown files with more information on GitHub.