Truth Trust Testimony: AI Disinformation and the Future of Evidence

The Godrej Lit Live! inaugural conversation — framed as “TRUTH. TRUST. TESTIMONY” — landed squarely on the fault lines of our information ecosystem: how we define truth, whom we trust to deliver it, and what counts as credible testimony in an age when synthetic media and algorithmic curation can manufacture conviction at scale. The panel’s observation that “AI has lowered the cost of disinformation” is not a rhetorical flourish; it describes a structural shift in how false narratives are created, distributed, and amplified — a shift that demands new rules of evidence, technology controls, and civic design if public discourse is to remain meaningful and resilient.

Background​

The inaugural session of Godrej Lit Live! assembled voices from science, media and digital communication: Nobel laureate Venki Ramakrishnan, Christian Stoecker — director of the Master’s in Digital Communication at Hamburg University — Booker Prize winner Shehan Karunatilaka, and moderator Anish Gawande. Their conversation clustered around three linked problems: what truth means in different spheres, how trust is built and eroded, and how testimony (photographs, recordings, eyewitness accounts, data) functions as evidence today.
That framing is timely. Independent red‑team audits and industry discussions show that modern chatbots and retrieval‑enabled assistants increasingly answer questions confidently while drawing from a web landscape that adversaries can pollute. This increases the chance that fabricated or low‑quality material will be treated by systems — and therefore by users — as credible evidence. The technical and social dynamics at play were front‑and‑center in the Lit Live! discussion and mirrored in contemporaneous research into AI‑targeted disinformation.

---

Truth: provisional, disciplinary, and under pressure​

Science, humanities and competing notions of truth​

Panelist Venki Ramakrishnan reminded listeners that scientific truth is provisional — it emerges from hypothesis, experiment, and consensus-building, always subject to revision when evidence changes. That humility is essential; it is also why scientific communication needs robust provenance and method disclosure so non‑specialists can judge degrees of certainty.
In contrast, the humanities and lived politics often treat truth as layered with interpretation and context. As Shehan Karunatilaka noted with wry candor, professions like advertising profit from persuasive storytelling; yet the stakes shift dramatically when a claim affects public health or civic choice. These distinctions matter for policy and product design: the same UX that serves quick, entertaining answers can be dangerously misleading when applied to high‑stakes topics.

Why AI matters for truth​

Two technical trends converge to threaten settled meaning in public debate:

• Retrieval‑augmented models increasingly pull live web content into answers. That improves recency but creates an attack surface for adversaries who seed machine‑digestible falsehoods across low‑quality pages and mimic outlets. When retrieval systems lack robust source‑quality discriminators, junk can masquerade as evidence.
• Helpfulness tuning — models are optimized to produce answers rather than refuse uncertain prompts. As refusal rates fall, confidently wrong answers rise. Independent audits have documented this trade‑off: systems that answer almost every prompt are more likely to repeat circulating false narratives if their retrieval or safety filters are insufficient.

These are not hypothetical: red‑team audits report marked increases in repeated false claims and a steep decline in refusals over measured intervals, demonstrating that product design choices materially alter factual risk.

---

Trust: the social shortcut and its vulnerabilities​

Trust is social, not purely informational​

The Lit Live! panel emphasized a core human truth: people rely on trusted intermediaries — family, journalists, doctors, community leaders — to simplify complex evidence. Trust is a cognitive shortcut that is essential in busy lives, but it becomes brittle when those intermediaries are replaced or impersonated by polished but synthetic content.
The problem is compounded when platforms treat engagement as the primary signal. Content optimized for attention is not the same as content optimized for truth. Algorithms prioritize what keeps users online; adversaries optimize content for virality, outrage, or machine readability. The result is an ecosystem where trust can be manufactured more cheaply than ever.

Real‑world evidence: how AI lowers cost for bad actors​

Independent reporting and monitoring show coordinated operations intentionally produce machine‑digestible content to game retrieval and ranking systems. By seeding many low‑traffic but crawler‑friendly pages, adding mimic formatting, and amplifying reposts, these operations can make false narratives appear more authoritative to automated systems. When chatbots pull from this noisy web without adequate provenance checks, the downstream effect is confident but false outputs.
Beyond webpages, synthetic audio and video have become easier to produce and distribute. Industry accords and technical proposals (watermarking, metadata tagging, provenance signals) have emerged to counter this trend, but those mitigations face practical limitations — watermarks can be stripped, and metadata can be lost during re‑sharing — so they are necessary but not sufficient defenses.

---

Testimony: what counts as evidence when media can lie?​

Photographs, recordings and the “Napalm Girl” problem​

Shehan’s question — does photographic evidence carry the same weight now as it did in the era of Pulitzer‑winning war photography? — is urgent. Once, a single iconic photograph could reshape public opinion because the labor and credibility cost of producing such images were high. Today, convincingly realistic images and audio can be synthesized with modest resources.
This means testimony must be re‑evaluated along two axes:

• Provenance: who created this content, where did it first appear, and what chain of custody can be demonstrated?
• Corroboration: can the content be independently verified by multiple, credible sources or by metadata that resists tampering?

Where provenance is missing or contested, the value of a single item as decisive testimony diminishes.

Platforms, UX and the testimony problem​

Design choices on platforms — whether to show source snippets, require explicit citations, or allow in‑line provenance indicators — materially affect how users interpret testimony. Product teams can push for evidence‑first interfaces that present claims as provisional and link to primary sources; failing that, users risk treating synthesized or unverified media as direct testimony.
Recent industry commitments to watermarking and labels recognize this, but technical workarounds and the volume of synthetic content mean policy and human processes must play equal roles.

---

Verifying the panel’s central claim: has AI really lowered the cost of disinformation?​

Short answer: yes — when measured by the ease, speed, and reduced resource intensity of producing plausible false content that spreads. Independent audits and investigations corroborate the panel’s intuition.

• Red‑team audits show a growing tendency for web‑connected assistants to repeat circulating false narratives, with refusal rates dropping as systems are tuned for responsiveness. That change makes it easier for a fabricated claim to be retrieved and presented as plausible evidence.
• Investigative reporting documents coordinated “grooming” operations that seed machine‑digestible falsehoods across numerous sites and mimics of legitimate outlets to game retrieval and ranking. Those operations are explicitly designed to be picked up by agents and chatbots.
• Industry-level mitigation talks — including watermarking and provenance standards discussed at cross‑industry meetings — implicitly confirm recognition by major players that synthetic content is now cheap to produce and potentially harmful if left unlabeled. But technical mitigations have limits, and adversaries continue to adapt.

These convergent data points support the panel’s claim: where creating credible disinformation once required journalists, broadcasters, or coordinated propaganda budgets, now generative models and automated distribution networks enable similar impacts with far lower cost and effort.

---

Strengths and limits of the evidence​

Strengths​

• Independent audits provide concrete measurements showing the direction of change: increased tendency to answer, increased exposure to low‑quality web signals, and documented cases where systems repeated fabricated narratives. Those findings are actionable for product and policy teams.
• Industry and platform conversations about watermarking, metadata, and provenance reflect a mature recognition of the problem and point to practical mitigations that can be deployed now.

Limits and caveats​

• Many audits use targeted, adversarial prompts (red‑teaming) rather than general accuracy tests. That means headline percentages characterize susceptibility to specific attack patterns rather than global correctness across all domains. Reported figures should be read as indicative of structural vulnerabilities, not as absolute performance metrics for every use case.
• Watermarking and metadata are not panaceas. They are helpful when maintained end‑to‑end, but they can be removed or degraded in downstream sharing, and they rely on broad adoption to be effective.
• Product tradeoffs matter. Systems tuned for engagement or helpfulness will continue to push the boundary between “answering” and “refusing,” and without governance and provenance layers, the default user experience favors closure over investigation.

When a claim is unverifiable — for example, a specific anecdote or unrecorded remark attributed to a public figure without corroboration — it should be flagged as such rather than amplified. The Lit Live! panel model of separating philosophy (what truth means) from operational practice (how testimony and trust are validated) is instructive here.

---

Practical implications for journalists, IT leaders and Windows users​

The Lit Live! discussion emphasized a shared responsibility: technologists must build safer systems; platforms must provide provenance; journalists must preserve unique reporting; and users must practice skeptical verification. Translating that into operational steps:

• For newsrooms and publishers:
• Publish machine‑readable provenance and canonical timestamps to aid automated verification.
• Invest in unique reporting and source ecosystems that are resilient to scraping and AI summarization.
• Require layered verification for high‑stakes claims and label AI‑assisted summaries explicitly.
• For IT leaders and Windows administrators:
• If deploying assistants (Copilot, Office AI, third‑party agents), enable conservative modes and require provenance for public‑facing outputs.
• Treat AI agents as privileged clients: log prompts, monitor source links, and apply human review for outputs that touch regulated domains.
• Use private retrieval (curated RAG) rather than the open web for sensitive corporate knowledge.
• For everyday users:
• Treat single‑turn AI answers as drafts, not finished facts; follow source links and prefer corroboration for consequential decisions.
• Look for provenance cues: is the content accompanied by original sources, timestamps, or author identifiers? Absence of those cues reduces credibility.
• Prefer tools and platforms that surface context and citations rather than black‑box summaries.

---

Technical and policy prescriptions​

The panel’s alarm is a call to action. The following recommendations map technical feasibility to governance need:

• Make provenance first‑class in retrieval stacks. Rank candidate sources not only by page rank but by long‑term reliability, authorship signals, and verifiable provenance. Conservative fallback logic should prefer refusal or human escalation over speculative answers on high‑risk queries.
• Require clear provenance UI in consumer assistants. Exposure of source snippets, direct links, and trust indicators reduces the chance users will accept synthesized claims uncritically. Product design can nudge users toward verification rather than false closure.
• Accelerate interoperable watermarking and content credentials, while acknowledging limits. Watermarks and metadata help detection and triage, but they must be complemented by legal and platform responses when manipulative content spreads.
• Fund and standardize independent adversarial monitoring. Red‑team audits that simulate real‑world misuse are valuable for vendor accountability and procurement decisions. Public, reproducible benchmarks help buyers and regulators make informed choices.
• Protect high‑value reporting economically. Licensing arrangements and technical endpoints (NLWeb, AutoRAG‑style APIs) can give publishers control over how their reporting is ingested by agents and can create revenue models that sustain investigative journalism.

---

Risks to watch and the likely near‑term trajectory​

• Adversaries will continue to shift tactics. As watermarking and provenance defenses improve, manipulative actors will attempt to degrade metadata, create better mimic sites, and use automated networks to amplify seeded content. Defenses must be adaptive and cooperative across platforms.
• Product incentives remain misaligned. Vendors that prioritize engagement and “helpfulness” risk amplifying falsehoods unless they adopt provenance and conservative fallbacks for risky topics. Independent audits show this trade‑off is real and material.
• The economic model for independent journalism is under pressure. Automated summarization and scraped content reduce referral traffic and ad revenue, making unique reporting more expensive and less sustainable. Policy and licensing innovations will matter if the journalism ecosystem is to survive the AI era.

---

Conclusion​

The Godrej Lit Live! panel distilled a worrying but actionable truth: generative AI and retrieval‑enabled assistants have materially lowered the cost of creating and distributing persuasive falsehoods. That structural change does not mean truth is dead; it means that the rituals and infrastructures that used to certify trust — editorial verification, provenance, legal accountability — must be retooled for a world where synthetic content is easy, fast, and accessible.
Technical fixes (watermarking, provenance signals, conservative retrieval) can blunt the immediate harms, but they must be paired with product design that surfaces uncertainty and with public investment in journalism, independent auditing, and cross‑platform threat intelligence. Individual users and enterprise IT teams also have concrete steps to reduce risk: require provenance, enable human review, and restrict web‑grounded retrieval for high‑stakes contexts.
The Lit Live! conversation makes a sober demand: if democratic discourse is to survive the era of cheap synthetic media, societies must redesign the trust infrastructure that underpins shared reality. That work will require engineers, editors, policymakers, and citizens to align incentives around evidence, provenance, and verification rather than mere engagement. The alternative is not only a proliferation of falsehoods, but a steady erosion of the civic goods truth and trust enable.

---

Source: Storyboard18 AI has lowered the cost of disinformation: LitLive!