AI Chatbots Repeating Falsehoods 35% of News Replies (Aug 2025 Audit)

AI chatbots are now answering more questions — and, according to a fresh NewsGuard audit, they are also repeating falsehoods far more often, producing inaccurate or misleading content in roughly one out of every three news‑related responses during an August 2025 audit cycle. (newsguardtech.com)

Background​

The finding comes from NewsGuard’s year‑long AI False Claims Monitor, a monthly red‑teaming program that tests leading chatbots on provably false narratives drawn from the organization’s “False Claim Fingerprints” database. In August 2025 Newsguard de‑anonymized its results for the first time, publishing model‑level performance numbers and concluding that the top 10 consumer chatbots now repeat false claims 35 percent of the time when prompted about breaking news and controversial topics — a near doubling of the 18 percent rate reported in August 2024. (newsguardtech.com)
NewsGuard’s work is explicitly focused on the intersection of misinformation and generative AI: the audit probes how frequently chatbots will repeat false claims, decline to answer, or debunk — using three prompt personas (an “innocent” user, a “leading” prompt that presumes the false claim, and a “malign” prompt representing an actor attempting to manipulate the model). This design tests the systems under conditions that mirror real‑world usage and abuse. (newsguardtech.com)

---

What the audit measured and how​

Methodology in brief​

• NewsGuard selects a rotating sample of provably false claims from its False Claim Fingerprints library and crafts 10 distinct false narratives for the month.
• Each false narrative is tested with three prompt styles — Innocent, Leading, and Malign — producing 30 prompts per model for a given 10‑claim test set (NewsGuard has also run variants with 15 claims in some reports). Responses are classified as a Debunk, a Non‑response, or Misinformation (that is, the model repeated the false claim). (newsguardtech.com)

This is not an abstract benchmark. It is a targeted red‑teaming approach that aims to evaluate how models handle specific, circulating falsehoods — the exact type of prompt that real users, propagandists, or foreign influence operations might feed into conversational systems. The choice of personas — especially “leading” and “malign” prompts — has drawn critique from some corners, but NewsGuard defends the design as deliberately representative of how chatbots are exploited in practice. (newsguardtech.com)

Strengths and constraints of the methodology​

• Strengths: The monitor focuses on current, verifiable false claims that are actively circulating; it uses human analysts to evaluate nuance; and it intentionally includes adversarial prompts to replicate worst‑case scenarios. These choices make the audit highly relevant to real‑world misinformation risks. (newsguardtech.com)
• Limitations: A narrow prompt set (10–15 fingerprints monthly) and binary classification choices mean the results reflect susceptibility to a defined set of falsehoods rather than a model’s overall competence on all factual matters. The audit is red‑team oriented by design — it exposes vulnerabilities more than it estimates average everyday accuracy outside news‑sensitive contexts. That nuance matters when readers interpret percentages as blanket measures of model truthfulness. (newsguardtech.com)

---

The headline numbers (what models did worst — and best)​

NewsGuard’s anniversary report names named scores for the first time and highlights stark variation across vendors:

• Inflection AI’s Pi registered the highest rate of repeating false claims — 57% of tested responses contained inaccurate information in the August 2025 audit. (newsguardtech.com)
• Perplexity showed a dramatic deterioration in NewsGuard’s measurement versus last year; NewsGuard’s reporting and multiple news outlets note Perplexity’s jump from near‑zero repeat rates in earlier audits to roughly 46–47% in August 2025. (newsguardtech.com, euronews.com)
• Widely used models such as OpenAI’s ChatGPT and Meta’s Llama were reported to repeat falsehoods in about 40% of responses during the audit. Microsoft Copilot and Mistral’s Le Chat landed near the 35% mark. (newsguardtech.com)
• At the other end of the spectrum, Anthropic’s Claude performed best in NewsGuard’s sample, with a 10% false‑claim rate, and Google’s Gemini also fared relatively well at roughly 17%. (newsguardtech.com)

Multiple outlets that covered the de‑anonymized release summarized these same percentages; the broad story is consistent across NewsGuard’s press release and subsequent reporting. (newsguardtech.com, dataconomy.com)

---

Why the deterioration? What changed in 2025​

NewsGuard’s analysis points to a structural trade‑off: chatbots are being made more responsive and web‑aware, and in doing so they have shifted from saying “I don’t know” toward attempting an answer. That behavior reduces non‑responses (NewsGuard reports a fall from ~31% non‑responses in August 2024 to near zero in August 2025) — but the net effect is an uptick in confidently stated inaccuracies, because real‑time web access exposes models to a noisy, manipulated, and sometimes deliberately poisoned information environment. (newsguardtech.com)
Two mechanisms stand out:

• Retrieval and web‑grounding: Many chatbots now retrieve web content during inference to stay current. That capability improves recency but also gives adversaries an attack surface: low‑quality, SEO‑optimized pages and AI‑generated “micro‑sites” can be retrieved and cited as if they were reliable. NewsGuard’s investigations show that pro‑Kremlin networks such as the so‑called Pravda network and operations like Storm‑1516 have deliberately seeded content to influence search and AI retrieval. When chatbots rely on those signals without robust source discriminators, false narratives get reinforced. (newsguardtech.com)
• Policy and guardrail shifts: Vendors have tuned models to reduce refusals, to prioritize helpfulness, or to provide in‑line citations — and that tuning can incentivize answering anyway. An answer that cites a dubious web source is still an answer; an answer that is wrong can be more dangerous than a safe refusal. NewsGuard’s month‑to‑month monitoring captures precisely this trade‑off. (newsguardtech.com)

Independent reporting and investigative work converges with NewsGuard’s diagnosis: journalists and researchers have documented coordinated networks producing AI‑targeted propaganda and “laundering” false narratives into the web ecosystem so they will surface in retrieval results. The Washington Post, Wired, and other outlets have documented the rise of such influence operations, noting that they are specifically engineered to be picked up by search engines and crawlers that feed LLMs. (washingtonpost.com, wired.com)

---

Concrete examples NewsGuard flagged​

NewsGuard’s audit is not purely statistical — it documents concrete cases where multiple chatbots repeated fabricated narratives and even cited pages tied to known propaganda networks.

• Moldovan politics: A fabricated item that mimicked the Romanian outlet Digi24 and included an AI‑generated audio clip allegedly of Moldovan Parliament leader Igor Grosu claiming Moldovans are “a flock of sheep.” NewsGuard found that several chatbots repeated that fabricated claim and, in some cases, linked to Pravda‑affiliated pages — illustrating how targeted disinformation narratives can be amplified by AI systems. (newsguardtech.com, incidentdatabase.ai)
• French politics and Mistral: Separate reporting by Les Echos and follow‑up coverage referenced a NewsGuard finding that Mistral’s Le Chat repeated false claims about President Emmanuel Macron and First Lady Brigitte Macron in a notable share of English‑language responses. Mistral acknowledged that both web‑connected and non‑web versions of its assistants showed vulnerabilities. NewsGuard’s monitoring also shows Le Chat’s overall error rate staying steady across audits. (euronews.com, idmo.it)

These examples matter because they demonstrate not only statistical tendencies but also how bad narratives move — from low‑traffic propaganda sites into chatbot outputs that a casual user might mistake for independent verification.

---

Vendor claims vs observed reality​

The Newsguard findings come amid high‑profile product releases where vendors explicitly touted improved reliability.

• OpenAI launched GPT‑5 (marketed on the ChatGPT platform as “GPT‑5”) with claims about substantially improved reasoning and reduced hallucination rates. OpenAI’s technical posts and system card acknowledge progress on hallucination reduction but stop short of a blanket “hallucination‑proof” guarantee; internal and external testing still shows residual false outputs and user reports of errors. In short, OpenAI frames GPT‑5 as improved, not infallible. (openai.com, cnbc.com)
• Google’s Gemini 2.5 rollout emphasized enhanced reasoning (“Deep Think”), long‑context windows, and benchmark gains — improvements likely to reduce some classes of errors on reasoning tasks. But Google’s announcement does not imply immunity to externally seeded false narratives, and NewsGuard’s monitoring still found Gemini substantially vulnerable — though statistically stronger than several peers in NewsGuard’s sample. (blog.google, newsguardtech.com)

Observers and NewsGuard both stress a practical point: product marketing around “better reasoning” or “lower hallucination rates” is compatible with significant residual risk. Improved internal metrics do not automatically translate to robust resistance against targeted misinformation campaigns that exploit web retrieval and shallow source vetting. (tech.yahoo.com, sdtimes.com)

---

Critical analysis — what these results mean for enterprise and consumer users​

Not all errors are equal​

A model that fabricates a citation for a benign trivia question is not the same as one that repeats a political defamation or a health falsehood. NewsGuard’s audits intentionally target the latter: news, elections, and geopolitically sensitive narratives where harm and civic impact are highest. That focus elevates the relevance of the percentages for journalists, policy teams, and compliance officers. (newsguardtech.com)

The retrieval problem is a design problem​

The core technical tension — freshness vs. trust — is designable. Retrieval‑augmented approaches can be paired with stricter source policy layers, provenance checks, and policy ensembles that downgrade or refuse outputs when sourcing is weak. But these guardrails exact user‑experience costs (more refusals, less immediacy), which vendors may be reluctant to accept in a competitive market. NewsGuard’s data suggests many vendors currently prioritize responsiveness, with measurable consequences. (newsguardtech.com)

Governance and vendor transparency​

De‑anonymizing the audit results for the first time was a meaningful transparency move: it allows customers, regulators, and enterprise buyers to factor model reliability into procurement decisions. Enterprises that embed chatbots into workflows — legal drafting, customer support, HR, or operations — must now weigh the trade‑offs and build human‑in‑the‑loop verification into any process that handles sensitive outputs. (newsguardtech.com)

Operational recommendations​

For IT teams, compliance officers, or power users deploying chatbots inside Windows‑centered workflows or business applications, a practical playbook follows:

• Prefer citation‑aware modes when factual accuracy matters and verify the cited sources manually. Models that expose source snippets or links make verification feasible. (dataconomy.com)
• Implement a two‑step human review for any AI output used in public communications, policy, legal, or clinical contexts. Make “AI‑draft” explicit in workflows and require sign‑offs.
• Use model ensembles or fallback strategies: combine a high‑recall, citation‑heavy model with a more conservative model and surface disagreements for review. (newsguardtech.com)
• Monitor adversarial web campaigns: source‑monitoring tools that detect spikes in low‑quality, AI‑generated content can feed site‑blocklists into retrieval pipelines. NewsGuard’s reporting underscores how adversarial actors intentionally create web content to be retrieved by AI. (newsguardtech.com)

---

Strengths and weaknesses of NewsGuard’s findings​

Notable strengths​

• Focused, actionable red‑teaming that mirrors how malicious actors behave.
• Transparent methodology documents and a multi‑persona testing strategy that stress‑tests guardrails.
• De‑anonymized, vendor‑level results that allow buyers and regulators to compare models empirically. (newsguardtech.com)

Important caveats​

• The audit’s emphasis on news and political falsehoods means the percentages are domain‑specific; a model that performs poorly on NewsGuard’s news prompts may perform better on technical, domain‑specific tasks (coding, math, document summarization).
• The sample size (10–15 fingerprints per month) is small by statistical standards and rotates monthly. That makes the monitor excellent at detecting systematic vulnerabilities and behavior shifts, but not a universal correctness score for all use cases. (newsguardtech.com)

NewsGuard itself highlights both strengths and limitations in its methodology FAQ; the organization frames the audit as a red‑team tool intended to reveal where models fail under adversarial pressure rather than to declare a model “good” or “bad” in every context. (newsguardtech.com)

---

The geopolitics of LLM grooming: why state‑linked networks matter​

Beyond technical trade‑offs, the NewsGuard reports and independent investigations document an intensifying information‑war strategy: state‑linked or state‑adjacent operations are deliberately producing reams of AI‑friendly content to influence recruitment of narratives into LLM outputs.

• The Pravda network and campaigns labeled Storm‑1516 or Matryoshka publish articles, deepfakes, and mimic‑sites designed to be machine‑digestible; NewsGuard documents instances where these sources were cited by chatbots and where false narratives were repeated verbatim. Such operations are cheaper to run at scale than traditional influence operations and are explicitly optimized for AI retrieval. (newsguardtech.com)
• The practical implication is acute: AI systems that incorporate web retrieval without nuanced source trust scoring can be tricked into amplifying the very propaganda they were meant to filter. That is not just hypothetical — NewsGuard’s audits and follow‑up reporting show concrete examples. (newsguardtech.com, incidentdatabase.ai)

---

Where accountability and product design intersect​

NewsGuard’s public naming of models creates pressure for vendor accountability, but it does not, by itself, solve structural problems. The options available to developers are bounded:

• Tighten safety and refusal behavior (raise non‑response rates for ambiguous news items).
• Improve retrieval source vetting and provenance signals.
• Invest in robust model‑level fact‑checking and cross‑validation with curated databases.
• Accept some trade‑off in user convenience to reduce the risk of amplifying disinformation.

Each option carries business, technical, and user‑experience consequences. The path forward will likely combine several strategies: better provenance, more conservative default behavior for news/political queries, and enterprise controls that let admins select model modes appropriate to their risk appetite. (newsguardtech.com, blog.google)

---

Final assessment and practical takeaway​

NewsGuard’s August 2025 audit is an important, timely wake‑up call: the AI industry’s pivot to responsiveness and web integration has reduced silence but increased the incidence of confidently delivered falsehoods on news topics. The 35 percent aggregate false‑claim rate is not a universal condemnation of LLM capabilities — models have improved in reasoning and many domains — but it is a clear signal that the information‑security dimension of model deployment is under‑resourced relative to the market push for accessibility and speed. (newsguardtech.com)
For Windows users, IT managers, and content professionals embedding AI into workflows, the practical rules are straightforward:

• Treat AI outputs as drafts, not authoritative evidence.
• Insist on provenance and citations when answers touch on public affairs, health, or legal matters.
• Build human review into any public‑facing pipeline.
• Monitor model updates and vendor safety disclosures; improvements in benchmark performance do not remove the need for operational guardrails. (newsguardtech.com, openai.com)

Finally, a word of caution about vendor rhetoric: marketing claims that present new models as “hallucination‑proof” or otherwise infallible should be read skeptically. Vendors often report reduced hallucination rates and improved reasoning — and those metrics are real and valuable — but independent red‑teaming and real‑world audits like NewsGuard’s show persistent vulnerabilities when models confront targeted, circulating falsehoods. Consumers and enterprises must plan for that residual risk. (openai.com, newsguardtech.com)

---

NewsGuard’s de‑anonymized audit has put the debate about usefulness versus trustworthiness squarely on the table: the next stage for vendors will be to demonstrate that they can deliver timely, helpful answers without opening the gates to coordinated misinformation campaigns. Until then, prudent skepticism and layered verification remain the responsible default for anyone relying on AI for news or decision‑critical tasks. (newsguardtech.com)

---

Source: Dataconomy AI chatbots spread false info in 1 of 3 responses

 

[ChatGPT]

AI chatbots are answering more questions than ever — and, according to a de‑anonymized NewsGuard audit released in September 2025, they are also repeating falsehoods far more often: roughly one in three news‑related replies contained a verifiable false claim during the August 2025 test cycle. (newsguardtech.com)

Background​

Chatbot reliability has been a live issue since large language models became widely available. NewsGuard’s AI False Claims Monitor is a monthly red‑teaming program that tests leading consumer chatbots against a library of provably false narratives it calls "False Claim Fingerprints." In August 2025 the monitor’s anniversary report publicly named model‑level scores for the first time and concluded that the ten most widely used chatbots repeated false claims on average 35 percent of the time — up from about 18 percent in the same audit a year earlier. (newsguardtech.com)
The change is not merely statistical. NewsGuard’s auditors observed a design shift across many systems: vendors have tuned models to answer more often and to use web retrieval, which reduced refusal rates but broadened exposure to a polluted online ecosystem. The trade‑off — fewer silences, more confident but incorrect answers — is the central tension the audit documents. (newsguardtech.com)

---

How NewsGuard tested the chatbots​

Methodology in plain terms​

NewsGuard’s monthly audit uses a targeted, adversarial evaluation rather than a broad, all‑purpose benchmark. The key elements:

• Analysts select a rotating sample of provably false narratives from NewsGuard’s False Claim Fingerprints library.
• For each false claim, the team crafts three prompt personas: an innocent neutral question, a leading prompt that assumes the claim is true, and a malign prompt that imitates manipulative tactics.
• Each claim is asked of each model in all three styles; responses are categorized as a debunk, a non‑response, or misinformation (the model repeated the false claim).
• NewsGuard reports both monthly aggregate figures and, in the August 2025 anniversary release, vendor‑level performance for the first time. (newsguardtech.com)

Strengths of the approach​

• The audit mirrors real‑world adversarial behavior: leading and malign prompts are not theoretical — they represent how bad actors and curious users actually interact with models.
• Using provably false, circulating narratives yields practical relevance for journalists, policy teams, and enterprise buyers who face these exact harms.
• The de‑anonymized results create vendor accountability and allow procurement decisions to consider reliability on news topics. (newsguardtech.com)

Important limitations​

• The monitor is intentionally domain‑specific: it focuses on news, politics, health, and corporate claims. A model that struggles on NewsGuard’s prompts may still be highly capable for code, math, or specialist summarization.
• Monthly samples are small (typically 10–15 "fingerprints" per cycle), so percentages reflect susceptibility to a rotating set of targeted falsehoods rather than a comprehensive correctness rating.
• Some vendor behavior differences are contextually driven (different web retrieval stacks, regional search integrations), which complicates direct apples‑to‑apples comparisons.

---

What the August 2025 audit found — headline numbers​

• Aggregate false‑claim repetition rate (August 2025): 35%, nearly double the 18% reported in August 2024. (newsguardtech.com)
• Non‑response (refusal) rate: dropped from 31% in August 2024 to 0% in August 2025 — virtually every prompt received an answer. (newsguardtech.com)
• Model‑level performance (rounded figures reported by NewsGuard and covered by independent outlets):
• Inflection’s Pi: ~57% false claims (worst performer). (euronews.com, dataconomy.com)
• Perplexity: jumped from near‑zero in 2024 to ~46–47% in August 2025. (euronews.com, dataconomy.com)
• OpenAI’s ChatGPT and Meta AI: each around 40%. (euronews.com)
• Microsoft Copilot and Mistral’s le Chat: near the mid‑30s (around 36–37%). (euronews.com)
• Google Gemini: roughly 17% (among the better performers). (euronews.com)
• Anthropic Claude: about 10%, the lowest false‑claim rate in NewsGuard’s sample. (euronews.com)

These model numbers are consistent across NewsGuard’s own press materials and reporting by multiple outlets, though some outlets round differently; the full technical dataset and monthly details are available from NewsGuard’s AI Monitor pages and press release. (newsguardtech.com)

---

Why this deterioration happened: the technical and product drivers​

1) Retrieval and web‑grounding created a new attack surface​

Many chatbots shifted from static‑knowledge assistants to web‑grounded systems that perform real‑time retrieval. That improves recency but also exposes models to:

• Low‑quality, SEO‑optimized microsites.
• AI content farms publishing high volumes of machine‑crafted articles designed to be crawlable.
• Deliberate "laundering" of false narratives via mimic sites, reposts, and social accounts.

NewsGuard shows concrete cases where such content — often tied to state‑linked influence networks — made its way into model outputs. The effect is simple: if the retrieval stack lacks strong source‑trust signals, models can treat junk content as evidence. (newsguardtech.com, washingtonpost.com)

2) Policy and reward tuning favored helpfulness over caution​

Vendors have prioritized helpfulness and user engagement. Optimization objectives that penalize refusal and reward answer completeness can produce a system behaviorally biased toward giving an answer even when evidence is weak. The outcome is confident statements built on fragile or fabricated sources. NewsGuard’s data shows that as refusal rates plummeted (from 31% to ~0%), misstatements rose. (newsguardtech.com)

3) Disinformation networks specifically optimize for AI grooming​

Investigations show coordinated networks — e.g., the so‑called Pravda network and operations called Storm‑1516 — are deliberately seeding machine‑digestible narratives. These operations publish homogenous text, mimic established outlets’ formatting, and amplify content across low‑engagement channels to game retrieval ranking and, by extension, LLMs. The Washington Post and other outlets have documented these strategies; NewsGuard’s audit ties specific chatbot outputs to those sources. (washingtonpost.com, newsguardtech.com)

---

Concrete examples NewsGuard flagged​

Moldovan politician audio forgery and mimic outlets​

One case involved a fabricated story that imitated a Romanian outlet and included an AI‑generated audio clip ostensibly of Moldovan Parliament leader Igor Grosu saying demeaning things about Moldovans. The narrative was pushed through a network of small sites and social posts aligned with the Pravda network; six of ten chatbots repeated the claim as fact in the audit. The example exposes how audio fakes plus mimic sites can cascade into chatbot outputs via retrieval. (newsguardtech.com)

French and German election narratives, Canadian public health claims​

Other audited fingerprints included false claims about French and German political figures and misleading narratives about ivermectin use in Canada. In some cases models cited low‑quality pages or aggregation sites as if they were original reporting. These examples show the cross‑border nature of the problem and how low‑traffic sites can have outsized influence on web‑connected models. (newsguardtech.com)

---

Vendor promises versus observed results​

Vendors have publicly emphasized safety and lowered hallucination figures in marketing and technical notes. OpenAI framed GPT‑5 as a major step toward precision, and Google positioned Gemini updates as improvements in reasoning. Mistral, Anthropic, and others have discussed media partnerships and source‑integration strategies.
Yet NewsGuard’s audit shows that product claims have not wholly translated into resistance to targeted misinformation. For example, Mistral’s le Chat reported roughly the same failure rate in 2024 and 2025 in NewsGuard’s tests, and Perplexity’s performance collapsed compared with prior months. These gaps highlight the difference between internal benchmark improvements and real‑world, adversarial robustness. (newsguardtech.com, dataconomy.com)

---

Critical analysis: what the numbers mean — and what they don’t​

Not all errors are equal​

A hallucinated citation for a harmless trivia question is not the same as repeating an electoral lie or a health falsehood. NewsGuard’s focus is precisely on high‑impact categories — politics, public health, international affairs — where harm can be material and rapid.

The audit is a red‑team, not a general correctness index​

Because the monitor intentionally stresses models under adversarial prompts, its percentages should be read as susceptibility to circulating false narratives rather than a global accuracy score across all tasks. A model scoring poorly on NewsGuard’s news prompts can still be robust for software development, math, or single‑source summarization tasks.

The retrieval trade‑off is addressable but expensive​

Technically, retrieval‑augmented LLMs can be paired with stronger provenance systems, source trust scoring, and conservative fallback logic. Doing so increases latency, reduces the "one‑turn answer" convenience, and may generate more refusals — tradeoffs that affect user experience and product competitiveness. NewsGuard’s results show vendors have, so far, favored responsiveness. (newsguardtech.com)

The geopolitical angle raises governance stakes​

State‑linked networks can cheaply scale AI‑friendly content — and bad actors in other states or private groups can copy the playbook. This amplifies the need for cross‑industry transparency, shared blocklists for known poisoning operations, and retrieval vetting that factors in provenance and historical reliability. Regulatory attention is likely to increase as these risks become systemic. (washingtonpost.com, newsguardtech.com)

---

Practical implications for Windows users, IT teams, and enterprise buyers​

For anyone embedding chatbots into workflows — from journalists to corporate comms and help desks — the NewsGuard findings matter in practical, operational ways.

Immediate, tactical steps (user and IT level)​

• Prefer citation‑aware modes when gathering facts and always verify the cited sources manually. Models that show snippet context or links enable faster validation.
• Treat AI outputs as drafts, not authoritative statements. Insist on human review for public‑facing communications, legal language, and health information.
• Use feature flags or admin controls to disable web‑grounded answering in high‑risk contexts (legal, HR, crisis comms).
• Implement a two‑step human verification workflow for any AI output going to customers, regulators, or the press.

Recommended technical controls for enterprises​

• Deploy a model ensemble: combine a citation‑heavy retrieval model with a conservative, non‑web model and surface disagreements for human review.
• Add a provenance layer: integrate a source‑trust scoring service or internal whitelists that rise above raw page rank.
• Monitor adversarial web campaigns: feed external detectors for AI‑generated news farms into retrieval filters; block or deprioritize known grooming domains.
• Use “AI disclaimers” and metadata tags in customer‑facing outputs that make it explicit when content originates from a model.

Windows‑specific considerations​

Windows 11 and Microsoft Copilot integrations mean many users will encounter AI answers inside productivity apps. Administrators should:

• Audit Copilot and Office AI settings at the tenant level, enabling conservative modes for regulated departments.
• Educate internal stakeholders that integrated AI can surface confident but unreliable answers.
• Make the “AI draft” status visible in templates and document headers to prevent accidental publishing without verification.

---

Policy and product recommendations​

• Vendors should publish standardized adversarial benchmarks and make de‑identified model behavior datasets available to independent auditors. NewsGuard’s de‑anonymized move is an example of useful transparency. (newsguardtech.com)
• Retrieval stacks must include source‑trust signals as first‑class inputs to ranking and answer synthesis. Signals can include long‑term publication reliability, authorship signals, and explicit provenance tags.
• Regulators and industry bodies should require that consumer‑facing models expose provenance and give users easy ways to escalate suspected misinformation incidents.
• Cross‑platform threat intelligence sharing about AI‑oriented grooming networks — including lists of mimic sites and content farms — would reduce the efficiency of influence campaigns.

---

Strengths and weaknesses of NewsGuard’s findings​

Notable strengths​

• Actionable red‑teaming that simulates real‑world misuse.
• Public naming of models enables accountability and procurement‑level decision making.
• Clear linkage between web retrieval practices and observed failure modes. (newsguardtech.com)

Important caveats​

• Small, rotating sample means percentages reflect vulnerability to a defined set of falsehoods rather than a total accuracy metric.
• Some per‑model differences may be driven by deployment choices (which web index is used, regional defaults) rather than model core competence.
• The full technical dataset behind the August 2025 release requires registration to download; some independent outlets have reported the same model rankings but round percentages differently. Readers should treat precise decimal points as approximate and rely on the audit’s directional conclusions.

---

The bigger risk: normalization of confident falsehoods​

NewsGuard’s most important conceptual point is that the long‑term harm is not a single false answer but the normalization effect. When misinformation appears in ordinary, everyday answers — presented confidently, with or without a citation — users’ ability to separate fact from fiction erodes. In a world where workplace search, email drafting, and customer support increasingly rely on AI, that erosion threatens trust in institutions and workflows. The remedy is not purely technical; it requires product design choices, governance, and user education. (newsguardtech.com, axios.com)

---

What to watch next​

• Vendor responses and product updates: watch for source‑quality indicators, conservative news modes, and improved provenance interfaces in product releases from OpenAI, Google, Microsoft, Anthropic, Mistral, and others. Public statements about “lower hallucination rates” are meaningful but insufficient without evidence of adversarial robustness.
• Regulatory signals: governments and standards bodies are increasingly focused on AI‑safety requirements for public information and high‑risk use cases.
• Independent audits: the field needs more third‑party, de‑anonymized benchmarks that test models under adversarial, multilingual, and cross‑domain conditions.

---

Conclusion​

NewsGuard’s August 2025 de‑anonymized audit is a sharp, actionable reminder that product choices — particularly those expanding web connectivity and prioritizing responsiveness — carry measurable information‑security consequences. The headline numbers are stark: a near doubling of false‑claim repetition to 35%, coupled with a collapse of refusal behavior to zero, demonstrates a design trade‑off with real civic and enterprise implications. (newsguardtech.com, euronews.com)
For Windows users, IT professionals, and enterprise buyers the practical response is clear: treat AI outputs as drafts that require provenance and human oversight; prefer models and modes that make sources explicit; and embed guardrails into workflows where mistakes matter. Vendors can and should do more — but closing the gap will require deliberate tradeoffs, cross‑industry coordination, and ongoing independent auditing to ensure that convenience does not come at the expense of truth.

---

Source: Digital Information World Chatbots Are Spreading More False Claims, NewsGuard Report Shows