Accuracy of Basic LLM Recall of LC Classifications

Accuracy of Basic LLM Recall of LC Classifications

Presentation at the 2025 CALA Canada Annual Conference

Vaillancourt, Shawn

Chinese American Librarians Association

2025-05-05

https://creativecommons.org/licenses/by/4.0/

application/pdf

eng

Text; Presentation

Recently, we had to work with collections data that had become disconnected from its LC classifications. Knowing that many LLMs have ingested a great deal of bibliographic metadata, but also wary of the possible mis-recollections of that data, we tested a few systems for feasibility with spot checks against Worldcat for accuracy. While the spot checks met our needs at the time, the question remained, how accurately can these systems recall LC classification on a “no frills” basis (i.e. without setting up a custom GPT with a Worldcat API or something similar). This proposal aims to present an intentional, methodical testing across the LC classification, to check the error rate in applying LC classifications, and whether certain classes present more of a challenge for LLMs to “recall”.