Taming AI Hallucinations: A Librarian's Guide to Verifiable Citations

Generative chatbots are increasingly creating work for human knowledge professionals: they answer confidently, invent citations and catalogue numbers, and send librarians on time-consuming hunts to prove that a referenced item never existed in the first place.

Background​

Generative large language models (LLMs) such as ChatGPT, Google Gemini and Microsoft Copilot were built to produce fluent, helpful text—but that same objective makes them prone to hallucinations: fabricated facts, invented quotations, and wholly imaginary sources. Independent audits and institutional reports show this is not a fringe problem. A multi‑market audit coordinated by public broadcasters found that a large share of assistant replies contained serious faults—sourcing problems, temporal staleness and outright fabrications—and that these failures appear across vendors and languages.
Libraries, archives and courts have begun to feel the operational cost of those hallucinations. Archivists report a rising volume of reference requests that appear to originate with AI, and several institutions—including the International Committee of the Red Cross—have publicly warned researchers that AI tools can invent archival references and catalogue numbers that have no basis in records.
This article synthesizes the reporting, audits and institutional reactions, explains why the problem exists, weighs the practical tradeoffs, and lays out a concrete playbook for librarians, archives, IT leaders and vendors. The coverage below cross‑checks audit data and institutional guidance drawn from multiple independent analyses and archival accounts to present a cautious, actionable view for information professionals.

---

How hallucinated citations reach librarians' desks​

The "prove‑negative" problem​

When an AI invents a unique primary source—an archival item, a manuscript accession number, or a journal issue that never existed—the burden falls on human experts to disprove the claim. That "prove‑negative" work is expensive. Archivists may need to search accession registers, finding aids and uncatalogued cartons; sometimes exhaustive searches still cannot give an absolute, provable negative quickly. This transforms single erroneous queries into hours of staff time. Institutional reports and journalist accounts describe archives triaging "phantom" research queries and adopting strict verification policies as a direct response.
A widely cited, but journalistic, estimate puts roughly 15 percent of some reference-desk email traffic as originating from AI tools—an indicator of the volume and not an official census. That figure has been repeated in reporting about state and university archives and is best treated as directional rather than definitive.

Real examples and legal fallout​

Hallucinated references are not only a nuisance; they have produced material harm. Courts in multiple jurisdictions have documented filings that cite nonexistent cases or authorities generated by chatbots, in some instances triggering sanctions or reprimands. One consumer tribunal cited a list of cases that turned out to be fabricated and explicitly labelled them "hallucinations" in its ruling. Those legal episodes illustrate how a confident, wrong citation can produce immediate professional and financial exposure.

---

Why models invent sources: the technical anatomy​

Language models are not search engines​

At their core, LLMs are optimized to predict the next token in a sequence, producing fluent text that sounds plausible. They are not, by default, verifiers. When a prompt requests a citation or source and retrieval signals are absent or weak, a model will often construct plausible-looking bibliographic information rather than refuse. That behaviour is structural: the generation objective rewards fluent completion, and many product designs favour answering over silence.

Retrieval‑Augmented Generation (RAG) and post‑hoc citations​

There are two main engineering patterns for providing facts and citations:

• Retrieval‑first (proper RAG) — the model retrieves documents from a curated index and conditions its answer on verifiable snippets. When implemented correctly with up‑to‑date indexes and explicit provenance, RAG materially reduces fabrication risk.
• Post‑hoc citation assembly — the model generates an answer and then produces citations afterwards, sometimes reconstructing plausible metadata that was never actually retrieved. This pattern opens the door to fabricated citations.

Many consumer deployments still use mix-and-match architectures or permissive post‑hoc citation pipelines. The audits show that these configurations are where citation hallucinations flourish.

Product design tradeoffs​

Vendors tune assistants for engagement and helpfulness—lower refusal rates and fuller answers. But that design choice raises the likelihood of "confidently wrong" outputs. Conservative defaults—declining to answer uncertain reference queries or explicitly marking uncertainty—reduce hallucination but also reduce perceived usefulness. The balance vendors choose has direct operational consequences for institutions and professionals.

---

What independent audits reveal (numbers and caveats)​

Multiple audits and journalistic inquiries converge on several core failure modes and provide quantitative snapshots:

• Roughly 45 percent of assistant replies in a public‑service audit contained at least one significant issue likely to mislead a reader.
• About 31 percent of answers had serious sourcing or attribution failures.
• Approximately 20 percent contained major factual or temporal errors.

Vendor comparisons vary by test and timing; for example, the same audit flagged one vendor for notably higher sourcing‑failure rates, but reported percentages differ slightly between summaries and over time as products evolve. These figures should be read as rigorous, editorially judged snapshots rather than immutable rankings.

---

Why libraries and archives are uniquely exposed​

Volume, variety and the uncatalogued remainder​

Public archives and special collections often contain large stores of uncatalogued material. A fabricated reference to a "unique" item therefore triggers an expensive manual search across registers, accession logs and uncatalogued material—work that offers no guaranteed closure. Several archives have started requiring requesters to disclose if a citation was produced by AI and to provide the raw prompt or model output to speed triage.

User expectations and information literacy​

Many users treat chatbots as a one‑stop authoritative source. That expectation is dangerous in research contexts because non‑expert users are unlikely to perform the bibliographic checks a librarian would. The mismatch between user trust and the model’s actual provenance-handling amplifies the operational burden on reference staff.

---

Institutional responses so far​

Archives, universities and some publishers have adopted pragmatic workarounds:

• Require disclosure of AI‑originated queries and the raw prompt/output as part of a reference request.
• Set published time budgets and limits on staff verification work for unverifiable AI leads.
• Pilot internal RAG tools and automated DOI/metadata checkers to triage likely valid leads faster.
• For high‑stakes or regulated contexts (legal, medical, compliance), adopt mandatory human‑in‑the‑loop signoffs and logging.

These responses accept that the scale and frequency of hallucinated leads make exhaustive manual verification unsustainable unless workflows change.

---

A pragmatic playbook for libraries and archives​

• Establish an AI disclosure policy
• Require researchers to state whether an AI was used to generate citations and to provide the full prompt and output.
• Publish the policy clearly on intake forms and FAQs.
• Set and publish verification time budgets
• Declare a standard, limited amount of staff time per request for verifying AI‑sourced leads; triage beyond that budget for high‑value projects only.
• Automate quick checks
• Use automated DOI resolvers, CrossRef/WorldCat checks and publisher APIs to rapidly validate or falsify bibliographic metadata. These tools can resolve many false leads before manual work is needed.
• Pilot a curated internal RAG index
• Build a locally curated discovery index for frequent collections and condition internal AI tools on that index to reduce fabrication risk.
• Teach users to verify
• Publish short, shareable guides that show how to check a DOI, query WorldCat, and use publisher pages. Embed verification steps into student and patron training.
• Track AI‑originated workloads
• Log when requests arise from AI outputs and quantify staff time spent triaging them. Use that data to inform resourcing and to justify changes to intake policy.
• Coordinate with legal and compliance teams
• For requests with legal risk, require counsel or certified reference verifiers before accepting AI‑generated authorities into official records. Courts have already shown they expect human verification.

This playbook treats AI as a source of leads, not as a secondary evidence layer, and puts the institution back in control of verification costs.

---

What vendors can and should do​

• Make retrieval‑first RAG the default for citation tasks and expose machine‑readable provenance (retrieved snippets, timestamps, DOIs, WorldCat links). Provenance must be native to output.
• Implement conservative citation modes that decline to invent—explicitly refuse to provide a citation when no verifiable result exists rather than fabricating metadata.
• Provide developer APIs for real‑time verification against CrossRef, PubMed, WorldCat and national archive catalogues so downstream services can confirm references before presenting them.
• Publish error‑rate reports by language and task, and support independent, rolling audits. Transparency will help institutions choose appropriate vendor integrations.

Those engineering and policy changes are achievable; they require vendor prioritization that may conflict with short‑term engagement incentives but will reduce downstream institutional costs and reputational harms.

---

Critical assessment: strengths, tradeoffs and systemic risks​

Strengths​

• Speed and discovery: LLMs can rapidly surface relevant keywords, authors, and topical structures that accelerate human search. Used as a brainstorming or discovery tool, they reduce time-to-insight in well-documented fields.
• Accessibility: For non‑specialists, assistants make dense material more approachable and can democratize basic research tasks.

Tradeoffs and risks​

• False authority: Confidently presented hallucinations look authoritative. For non‑expert users and in contexts without mandatory verification, those hallucinations can propagate into scholarship, legal filings and the public record.
• Resource drain: Without institutional controls, libraries risk diverting scarce staff time to chasing phantom items—work that reduces capacity for genuine research support.
• Scale amplification: Even modest hallucination rates translate into large absolute volumes when assistants are used by hundreds of millions of people; that scale makes ad hoc responses insufficient.

Where improvement is realistic​

The core failure modes are engineering and product issues—no metaphysical barrier prevents better provenance exposure, conservative citation defaults, or more robust RAG pipelines. Independent audits show that retrieval‑first designs with explicit provenance materially reduce hallucination risk. The open challenge is aligning commercial incentives with public-interest accuracy so vendors implement these mitigations widely and by default.

---

Flags and cautions about the evidence​

• The 15 percent figure about reference emails originating from AI is reported in journalism and institutional anecdotes; it should be treated as directional rather than a definitive, system‑wide statistic. Institutions cite different empirical snapshots and local workloads vary.
• Vendor performance metrics change rapidly with model updates. Audit snapshots are valuable but time‑bound; repeated, independent monitoring is required to track progress or regressions. Published percentages for individual products (for example, sourcing failure rates) differ slightly across summaries and should not be taken as immutable rankings.

These cautions underscore why archives and libraries must adopt operational policies now rather than wait for vendor perfection.

---

Practical checklist for Windows IT teams and enterprise managers (short version)​

• Audit and lock down AI settings in productivity software integrated with Windows (e.g., disable unconstrained web retrieval for general users).
• Enforce DLP and endpoint protections to prevent uploading of sensitive data to public chatbots.
• Require human sign‑off for AI‑generated material touching legal, financial or patient‑facing content.
• Log AI interactions (prompts, model version, timestamps) for auditability and post‑incident analysis.

---

Conclusion​

Generative AI is reshaping how people seek information; its real value lies in assisted discovery, not as a standalone verifier. Libraries and archives are now contending with a new operational drag: time spent proving that AI-sourced citations do not exist. The emerging institutional fixes—AI disclosure policies, verification time budgets, automated DOI checks and curated internal RAG indices—are practical and implementable.
Vendors can and should make structural changes: default to retrieval‑first designs for citation tasks, expose provenance natively, and provide conservative refusal modes when evidence is absent. Regulators, publishers and professional bodies should push for independent audits and machine‑readable provenance standards so that institutions can trust the chains of evidence that underpin AI outputs.
The underlying technical problem is tractable; the harder work is organizational and economic. Until vendors, institutions and users realign incentives toward verifiability, librarians and archivists will continue to shoulder the downstream burden of correcting AI's confidently delivered mistakes. The pragmatic path forward is clear: treat AI outputs as ledgers of leads to be verified, not as a substitute for provenance, and build the workflows and tooling that make verification fast, auditable and sustainable.

---

Source: Popular Science Librarians can’t keep up with bad AI