The short answer is: no — not yet. Recent consumer head‑to‑head tests, vendor release notes and independent audits show clear progress: hallucinations are less frequent in many flagship models, and some systems now ship with retrieval and provenance features that reduce certain classes of errors. But the problem has not disappeared; it has evolved. Hallucinations are rarer in some scenarios, more likely in others, and — critically — the trade‑offs vendors make to increase responsiveness and coverage sometimes raise the risk of confidently delivered falsehoods. The result is an uneasy middle ground for users and IT teams: smarter assistants that still require verification, governance, and design choices to avoid costly mistakes.
AI “hallucinations” — the tendency of generative models to invent facts, misattribute quotes, fabricate sources, or produce internally inconsistent answers — have been a live operational risk since large language models entered mainstream use. Vendors have responded through model improvements, retrieval‑augmented architectures (RAG), tool chains and “thinking” or multi‑mode routing that attempt to distinguish when a short answer suffices and when deeper reasoning or external verification is required. OpenAI’s public launch of GPT‑5 and Google’s Gemini 2.x family are the most visible recent steps in that direction, and they are explicitly marketed as both smarter and less error‑prone than their predecessors. These vendor claims are backed by internal benchmarks published in product notes, and also challenged by independent red‑teaming and information‑reliability audits that show mixed results.
At the same time, recent independent audits show a worrying emergent pattern: as chatbots respond to more live web queries and decline less often, the proportion of confidently delivered false claims in news‑related prompts increased — a dynamic traced to the polluted and adversarial state of parts of the web. That trade‑off — fewer refusals, more confident mistakes — is central to whether we can claim to be “past” hallucinations. It’s a trade‑off worth understanding before embedding these systems in production workflows.
Why this matters: small tests like the one Digital Trends ran are valuable for practical diagnostics — they show how product updates and UI choices change the day‑to‑day experience. But they are limited in scope and easily affected by transient factors: model version (e.g., flash vs. deep‑thinking mode), prompt phrasing, whether the assistant used a web retrieval plugin, time of day, and even slight wording changes. A flawless 10‑question run does not mean a system is hallucination‑proof; conversely, one slip‑up in ten questions does not mean the model is irredeemably unreliable. The key takeaways are behavioral and operational, not absolute.
At the same time, independent monitors paint a different picture on the most consequential front: current‑events and news queries. NewsGuard’s monthly AI False Claims Monitor reported that, in August 2025, the ten leading consumer chatbots returned verifiably false claims on news prompts roughly 35 percent of the time — almost double the rate a year earlier. The audit’s central observation is instructive: systems that reduced non‑responses and answered more questions tended to pull from a polluted web corpus and therefore amplified falsehoods more often. This is the asymmetry: lowered refusal rates improve responsiveness but expose models to the web’s misinformation economy.
Two implications follow:
The responsible posture for users and IT teams is pragmatic caution: take advantage of improvements, but build verification, provenance, and human oversight into every pipeline where accuracy matters. Small consumer tests like the Digital Trends piece are useful—they show the user experience and illustrate improvements—but they must be read alongside independent audits and vendor documentation before making high‑stakes trust decisions.
Source: Digital Trends Are we finally past the AI hallucination problem? I put the top AIs to the test
AI “hallucinations” — the tendency of generative models to invent facts, misattribute quotes, fabricate sources, or produce internally inconsistent answers — have been a live operational risk since large language models entered mainstream use. Vendors have responded through model improvements, retrieval‑augmented architectures (RAG), tool chains and “thinking” or multi‑mode routing that attempt to distinguish when a short answer suffices and when deeper reasoning or external verification is required. OpenAI’s public launch of GPT‑5 and Google’s Gemini 2.x family are the most visible recent steps in that direction, and they are explicitly marketed as both smarter and less error‑prone than their predecessors. These vendor claims are backed by internal benchmarks published in product notes, and also challenged by independent red‑teaming and information‑reliability audits that show mixed results. At the same time, recent independent audits show a worrying emergent pattern: as chatbots respond to more live web queries and decline less often, the proportion of confidently delivered false claims in news‑related prompts increased — a dynamic traced to the polluted and adversarial state of parts of the web. That trade‑off — fewer refusals, more confident mistakes — is central to whether we can claim to be “past” hallucinations. It’s a trade‑off worth understanding before embedding these systems in production workflows.
What the small consumer test found — and what it doesn’t prove
Digital Trends ran a small, practical experiment: five consumer chatbots (Google Gemini, ChatGPT, Grok, Deep AI, and Microsoft Copilot) were each asked the same set of ten unambiguous trivia questions. The goal was deliberate: choose items with single, verifiable answers so the output could be classified as correct or incorrect without ambiguity. The headline result: most models answered the majority of questions correctly; Google Gemini and ChatGPT were reported to return all correct answers in that sample, while two of the five systems produced at least one error. The tester also recorded whether models supplied sources spontaneously or only after prompting. This kind of consumer test is useful as a snapshot of user experience, but it is not a substitute for controlled benchmarking or adversarial audits.Why this matters: small tests like the one Digital Trends ran are valuable for practical diagnostics — they show how product updates and UI choices change the day‑to‑day experience. But they are limited in scope and easily affected by transient factors: model version (e.g., flash vs. deep‑thinking mode), prompt phrasing, whether the assistant used a web retrieval plugin, time of day, and even slight wording changes. A flawless 10‑question run does not mean a system is hallucination‑proof; conversely, one slip‑up in ten questions does not mean the model is irredeemably unreliable. The key takeaways are behavioral and operational, not absolute.
The technical reality: hallucinations are decreasing in some measured metrics — but not eliminated
Vendors are publishing explicit metrics showing big relative improvements on internal and public benchmarks. OpenAI’s published model tables for GPT‑5 indicate substantially lower hallucination rates on several benchmark suites compared with older models; the company also introduced a multi‑mode routing approach that chooses between a fast answer and a deeper “thinking” variant based on complexity. Those concrete numbers matter: they show progress, especially on long‑form factual benchmarks where reasoning stability and tool use improve performance.At the same time, independent monitors paint a different picture on the most consequential front: current‑events and news queries. NewsGuard’s monthly AI False Claims Monitor reported that, in August 2025, the ten leading consumer chatbots returned verifiably false claims on news prompts roughly 35 percent of the time — almost double the rate a year earlier. The audit’s central observation is instructive: systems that reduced non‑responses and answered more questions tended to pull from a polluted web corpus and therefore amplified falsehoods more often. This is the asymmetry: lowered refusal rates improve responsiveness but expose models to the web’s misinformation economy.
Two implications follow:
- Benchmarks and vendor release notes show clear technical progress in controlled conditions and for many use cases (e.g., coding, math, and long‑context summarization).
- Real‑world, adversarial, or rapidly evolving information contexts still produce error rates that matter for news, legal, and medical domains.
Why hallucinations still happen (brief primer for IT decision‑makers)
- Probabilistic generation: Large language models approximate text patterns. When asked for a fact outside their training distribution or when retrieval fails, they may generate plausible but false content.
- Retrieval and source quality: RAG architectures improve factuality if the retrieval layer returns credible sources. If retrieval harvests low‑quality sites, the model can confidently repeat misinformation.
- Tool integration and latency: Using web retrieval or external tools reduces some hallucinations but introduces dependencies (APIs, crawl windows, ranking biases) that can themselves be attack vectors.
- Optimization trade‑offs: Vendors tune models for helpfulness and responsiveness. This often reduces refusals but increases the chance of producing a fast plausible — and sometimes wrong — answer.
- Prompting and adversarial inputs: Carefully crafted prompts or “prompt injections” can steer models to hallucinate or to reveal hallucinated content with high confidence.
Cross‑referenced evidence: what independent audits and vendor notes actually say
- NewsGuard’s August 2025 audit found a 35 percent false‑claim rate across ten leading chatbots on news‑related prompts, an increase tied to the industry’s shift toward web retrieval and lower refusal rates. That audit is an independent, adversarial test that specifically targets circulating news falsehoods.
- OpenAI’s public GPT‑5 announcement and release notes stress both improved capability and explicit reductions in measured hallucination rates on internal benchmarks. The GPT‑5 product page describes a unified system with “thinking” and fast modes; the model release notes include tables comparing hallucination rates across model families. These published metrics indicate significant improvements on several standardized tests, though they are vendor‑posted figures and must be read against independent audits.
- Multiple third‑party analyses and bench tests (academic papers and platform benchmarks) show mixed results: some models perform better on structured factual tasks, others still produce fabricated citations or err on domain‑sensitive items like legal or medical recommendations. Those studies reinforce the nuance: gains exist, but failure modes remain and are contextual.
Practical takeaways for Windows users, IT admins and power users
- Treat AI outputs as drafts, not authoritative evidence. Even models with low headline hallucination rates generate errors on edge or time‑sensitive queries.
- Insist on provenance. Prefer chatbots and deployments that expose retrieval links, timestamps, and source context. If a model cannot produce a credible source, require manual verification before operationalizing the output.
- Design human‑in‑the‑loop verification for high‑stakes decisions. For legal, medical, compliance, or financial tasks, require a named human reviewer and a documented verification trail.
- Use model routing intentionally. Configure product settings that choose deeper “thinking” or retrieval modes for complex tasks and fast modes for low‑risk conversational work.
- Monitor and log. Capture model outputs, their source lists, and any downstream actions; this makes audits and rollbacks possible.
- Keep offline authoritative sources for critical queries. For internal knowledge or regulated content, connect models to private, curated corpora rather than open web retrieval.
- Educate users. Training on prompt design, recognition of spurious citations, and verification workflows reduces downstream harm.
A quick fact‑check of the consumer test’s trivia (verified references)
The Digital Trends test used straightforward trivia whose answers are widely documented. Verifying those items is useful because it highlights the difference between easy, closed‑world questions and open, adversarial news prompts.- Moon landing: Neil Armstrong became the first human to step onto the lunar surface during Apollo 11; the landing touched down July 20, 1969 (UTC landing time varies and some records mark Armstrong’s step as early on July 21 UTC). NASA’s Apollo 11 mission pages provide the canonical record.
- First woman to win a Nobel Prize: Marie Curie was the first woman to be awarded a Nobel Prize (Physics, 1903, shared) and later won the 1911 Nobel Prize in Chemistry; this fact is recorded by NobelPrize.org and major encyclopedias.
- Only sea without coastlines: The Sargasso Sea is conventionally described as the only named sea in the world without land borders; it is bounded by ocean currents rather than coastlines.
- Renaissance artist buried in Rome’s Pantheon: Raphael (Raffaello Sanzio) requested burial in the Pantheon; his sarcophagus and memorial are located there and have been the subject of scholarly coverage and public records.
- Year the United Nations was established: 1945, following the UN Charter signed in June 1945 and the organization’s founding in its modern form that year. (Widely documented; vendor and encyclopedic sources confirm 1945.)
- Country that drinks the most coffee per capita: Finland is consistently reported as the top coffee consumer by per‑person kilograms in global per‑capita rankings compiled by agencies and market research; multiple sources cite Nordic countries topping the lists. Use recognized datasets (International Coffee Organization, Statista) for exact kg/per capita numbers.
- Rarest / most expensive spice by weight: Saffron is widely cited as the most expensive spice per kilogram due to labor intensity and low yield. Price ranges vary with quality and origin, but saffron commonly appears at the top of “most expensive spice” lists.
- Character both Robert Downey Jr. and Benedict Cumberbatch have played: both actors have portrayed versions of Sherlock Holmes (Downey in feature films, Cumberbatch in the BBC series), a straightforward casting fact.
Strengths, risks and the operational balance
Strengths we can credit:- Measurable progress: Newer model families have demonstrable reductions in hallucination rates on many standard benchmarks and real‑task evaluations. Vendor engineering has clearly improved calibration, retrieval, and multi‑mode routing.
- Better tooling for verification: Many consumer and enterprise assistants now include retrieval links, citation toggles, and plugin ecosystems that allow source inspection when enabled.
- Practical accuracy wins: For structured tasks — coding, formulaic reasoning, long‑document summarization — improvements are substantial and materially helpful in many workflows.
- Confidence without provenance: The most dangerous failure mode is a model asserting a falsehood confidently and without credible sourcing. Independent audits show this happens often on news and fast‑moving topics.
- Adversarial and web‑sourced errors: Pulling from the live web exposes models to manipulative content farms and coordinated disinformation campaigns; lowering refusal rates increased false positives in some independent measurements.
- Domain sensitivity: Legal, medical, and compliance use cases still need defensible chains of evidence; hallucination reductions on generic benchmarks do not translate into guaranteed safety in regulated contexts.
- Operational complexity: More capable, multi‑mode models introduce cost and governance complexity (thinking‑mode compute cost, tooling for source vetting, quotas and throttles).
Hard recommendations for WindowsForum readers and IT teams
- Require provenance for public‑facing answers. Only publish or act on AI‑generated facts that are accompanied by verifiable source links.
- Tier AI uses by risk. Automate low‑risk tasks (summaries, drafting) but require manual sign‑off for legal, financial, health, or operational controls.
- Lock down retrieval for regulated corpora. Use private RAG with curated documents for sensitive information instead of open web retrieval.
- Monitor performance and audit regularly. Run an internal AI‑reliability audit aligned to your business domain at least quarterly.
- Train staff. Make prompt literacy and verification part of role onboarding and performance metrics.
- Use “thinking” modes intentionally. Route complex queries to deeper reasoning variants, but set clear cost and verification rules for their outputs.
Final assessment: a realistic verdict
The industry has reduced hallucinations in measurable ways, especially in controlled benchmarks and many production tasks. Vendors are shipping models with explicit “hallucination reduction” engineering and new operational features that improve factuality. However, independent adversarial audits — notably those targeting current events and news prompts — reveal an important countercurrent: responsiveness plus web retrieval can increase confident falsehoods. That means we are not “past” hallucinations; we are navigating a complex transition where the symptom is changing, not disappearing.The responsible posture for users and IT teams is pragmatic caution: take advantage of improvements, but build verification, provenance, and human oversight into every pipeline where accuracy matters. Small consumer tests like the Digital Trends piece are useful—they show the user experience and illustrate improvements—but they must be read alongside independent audits and vendor documentation before making high‑stakes trust decisions.
Appendix — Quick research references used to verify key claims
- NewsGuard’s August 2025 AI False‑Claims Monitor (audit showing ~35% false claims on news prompts).
- OpenAI’s GPT‑5 release notes and model tables documenting hallucination metrics and “thinking” mode.
- Vendor / community discussion and observed rollout notes describing GPT‑4.5 → GPT‑5 progression and commercial positioning.
- Canonical facts used in the Digital Trends trivia test: NASA/Apollo 11 records for moonwalk date and Neil Armstrong; NobelPrize.org / Britannica for Marie Curie; Britannica and encyclopedic sources for the Sargasso Sea and Raphael; major market data and AP/Statista reporting for Finland’s coffee consumption; multiple commodity‑market references for saffron as the world’s priciest spice by weight.
Source: Digital Trends Are we finally past the AI hallucination problem? I put the top AIs to the test