The National Cyber Security Centre’s 2025 Annual Review delivered a blunt verdict: the UK’s cyber threat environment has escalated from episodic nuisance to sustained national emergency, and the question for leaders is no longer whether they will be attacked but how they will survive the attack when it happens. The Review shows the NCSC handled 204 “nationally significant” incidents in the 12 months to August 2025 and managed 429 incidents in total — an average of roughly four serious incidents a week — and pairs those figures with a pointed call to action from ministers urging CEOs to make cyber a board-level priority. These numbers are not abstract; they track through to real-world collapses of availability, supply chains, and consumer trust at household-name firms and to near-catastrophic risks for Critical National Infrastructure (CNI).
It is crucial to stress the difference between demonstrated capabilities and speculative futures. Some scenarios — polymorphic AI malware that rewrites itself for indefinite evasion — are technically plausible and must be prepared for as risk scenarios. Other claims that specific strains of commodity malware already operate as fully autonomous, goal‑seeking agents remain conditional and sometimes unverified. Good defence planning uses credible, observable trends while flagging unverified claims with care.
The path forward is pragmatic and organisational, not purely technological. Trusted partnerships with vendors who can demonstrate audited, testable AI safety controls; cross‑sector intelligence sharing that turns indicators into collective shields; and a board that treats cyber as a strategic, measurable risk will turn the NCSC’s urgent warning into an operational advantage. The future resilience of the UK’s critical systems will be determined by the speed at which leaders convert this warning into governance, funding and disciplined execution.
Source: The AI Journal Cyber Resilience and AI Risk: Safeguarding the UK’s Critical Infrastructure in a New Threat Landscape | The AI Journal
Background
The NCSC’s warning: scale, severity and a new normal
For decades the NCSC’s role has been technical but supportive; in 2025 its Annual Review reframed the organisation’s message into strategic urgency. The Review documents a 130% year‑on‑year rise in nationally significant incidents and a 50% increase in the most severe events — metrics that transformed “cyber” from an IT risk into a systemic economic and social risk. The report explicitly links recent retail disruptions, manufacturing stoppages and large-scale data compromises to cascading business and public‑service harms, and it makes a governance demand: cyber resilience must be owned at board level.How common is a breach?
The empirical background to the national alarm is stark: UK government statistics show that just over four in ten businesses (43%) reported any kind of cyber breach or attack in the last 12 months, equating to hundreds of thousands of affected organisations. The same survey documents differences by size — medium and large businesses report substantially higher prevalence — underscoring that visibility, complexity and reporting practices matter when we interpret headline percentages. This is not a minor corporate governance problem; it’s an economic resilience problem.What recent attacks exposed
Retail and manufacturing shocks that read like a worst-case scenario
High‑profile attacks in 2025 against household brands accelerated public attention. Marks & Spencer reported severe online disruption and large revenue impacts; Co‑op confirmed customer data access and significant operational headwinds; Harrods temporarily restricted online services; and Jaguar Land Rover (JLR) suffered production halts that reverberated across supplier networks. These incidents illustrate the common pattern the NCSC documented: ransomware, credential theft and supply‑chain disruption have migrated from targeted data theft to operational sabotage with tangible effects on physical goods and services. Reporting shows retailers experienced both direct lost sales and sustained reputational damage, while JLR’s disruption cost estimates — reported by multiple outlets — ran into the hundreds of millions and, by some independent estimates, approached £1.9–£2.0 billion when knock‑on supply chain impacts are included. Those figures have real consequences for employment, local economies and national GDP.The supply-chain multiplier
The JLR case is illustrative: an attack on a single OEM can ripple down tiered suppliers that operate on narrow margin and tight inventory models. When factory IT systems go offline, parts flow stops, invoices and payments delay, and smaller suppliers’ liquidity evaporates — a classic systemic risk trajectory the NCSC cautions about. In JLR’s case, government intervention in the form of emergency finance and guarantees highlighted how cyber risk can rapidly become a question of national industrial policy.AI: a force multiplier for defenders and attackers
Defensive AI — faster triage, smarter governance
Artificial intelligence is central to the resilience argument in the NCSC Review and in commercial practice. Security teams increasingly deploy ML‑based anomaly detection, automated triage, and AI assistants to run playbooks at machine speed. Microsoft’s Security Copilot and Purview capabilities are concrete examples of how defenders are using AI to reduce mean‑time‑to‑detect and mean‑time‑to‑contain: Purview adds data governance and prompt/response controls to limit accidental data leakage into generative systems, while Security Copilot integrates signals from Defender, Sentinel and Purview to speed investigations and to produce natural‑language summaries that expedite decision making in SOCs. Those integrations are being adopted to help organisations keep pace with the compressed windows of risk. The benefits are tangible:- Faster forensic triage and incident summarisation.
- Automated enforcement of data‑handling policies for AI prompts.
- Integration across identity, endpoint and data layers to prioritise response.
Offensive AI — scaling deception and autonomy
At the same time, threat actors are incorporating AI into their toolkits. AI enables highly personalised social engineering at scale, automated prompt attacks, polymorphic payloads and faster lateral movement strategies. Deepfakes, voice‑spoofing and AI‑generated phishing messages have already been used in executive‑targeted fraud and extortion campaigns. The criminal adoption of reinforcement learning and automated decision guards raises the spectre of malware that adapts its tactics in real time to avoid detection and achieves higher success rates in credential theft and lateral escalation.It is crucial to stress the difference between demonstrated capabilities and speculative futures. Some scenarios — polymorphic AI malware that rewrites itself for indefinite evasion — are technically plausible and must be prepared for as risk scenarios. Other claims that specific strains of commodity malware already operate as fully autonomous, goal‑seeking agents remain conditional and sometimes unverified. Good defence planning uses credible, observable trends while flagging unverified claims with care.
Anthropic, “Claude Plays Pokémon” and the debate about AI planning
Anthropic’s public experiments with Claude — including the well‑documented “Claude Plays Pokémon” livestream and later demonstrations of longer‑duration agentic behaviour — have become a focal point in the debate about what current LLMs can actually plan and whether their internal “reasoning” maps to goal‑directed agency. Reporting across mainstream outlets showed Claude making extended, multi‑step plans in a constrained gaming environment and, in higher‑capability models, sustaining those plans longer than previous iterations. Those demonstrations are useful diagnostics: they give researchers a sandbox in which to probe memory, planning, and stepwise reasoning in large models. But the exercise is dual‑edged. Anthropic’s red‑teaming and constitutional AI research also shows how LLMs can be redirected via hidden instructions or prompt‑injection techniques. The “Claude Plays Pokémon” project was designed to explore capabilities, but it also revealed how model behaviour can be coaxed or misdirected with unexpected inputs. This is precisely why security teams worry about embedding generative agents in critical systems: even minor unanticipated behaviours in a toy environment can translate into dangerous deviations when the same models control or influence operational processes. Scholars continue to debate whether stepwise explanations from LLMs reflect true internal planning or are post‑hoc rationalisations built from training data. The result is practical: defenders cannot rely solely on an LLM’s internal “explanations” for safety assurances. Organisations must demand external, testable assurances — scenario tests, red‑team results, reproducible audits and rigorous guardrails — before placing generative agents where they touch CNI operations.What CNI operators must change — a practical playbook
The stakes for Critical National Infrastructure are existential. Networks that support transport, energy, water, health and financial systems cannot accept brittle experimentation without engineered safety. The following playbook is intentionally operational and board‑readable.1. Governance: make cyber a board responsibility
- Adopt the NCSC Cyber Governance Code of Practice and require board‑level cyber training. The government’s recent ministerial letter explicitly instructs major companies to treat cyber risk as a board issue and to sign up for NCSC services such as the Early Warning service. This is not symbolic — the letter defines immediate actions for directors that materially reduce enterprise risk.
- Set measurable resilience KPIs: mean‑time‑to‑detect, time‑to‑contain, time‑to‑recover and business recovery objectives.
2. Identity and access: treat identity as infrastructure
- Enforce phishing‑resistant MFA (passkeys/FIDO2) for all high‑privilege accounts.
- Transition privileged access to ephemeral credentials and strong PAM controls.
- Audit service accounts, API tokens, and agent privileges continuously.
3. Data governance for AI: use DSPM and DLP controls
- Deploy Data Security Posture Management (DSPM) and DLP for AI prompts and agent responses; Microsoft Purview offers concrete capabilities to prevent data oversharing to Copilot instances and to detect risky AI usage patterns. Those tools should be integrated with SOC workflows to provide forensic traces of agent‑driven activity.
4. Resilient architecture and recovery
- Build parallel recovery workflows for critical OT/IT services, with isolated backups and rehearsed runbooks.
- Adopt network segmentation between IT and OT, with strict egress controls for controllers and safety systems.
- Ensure offline, immutable backups and rehearsed DR exercises.
5. Threat intelligence and collective defence
- Participate in sector information‑sharing bodies and subscribe to NCSC Early Warning and takedown services.
- Contribute indicators and playbooks to secure cross‑sector threat feeds; the NCSC highlights collaboration as central to containment.
6. Rigorous supplier and supply‑chain controls
- Make Cyber Essentials (and equivalent baseline standards) mandatory in procurement.
- Require suppliers to demonstrate incident response, backup separations, and verified insurance posture.
7. People and exercises
- Run cross‑functional crisis exercises (board, legal, communications, HR, technical) quarterly.
- Build a resilience culture that encourages early reporting — surveys show many staff hesitate to flag incidents. Direct cultural fixes (no‑blame reporting, anonymous channels) reduce detection delays and limit damage.
Cost, capability and trade‑offs
AI‑enabled security can shorten containment windows, but it does not come free. Organisations are seeing:- Rising telemetry and storage costs as AI systems ingest and retain more logs and prompts.
- Increased vendor lock‑in risk when adopting managed AI SOC services.
- Higher specialist headcount requirements to operationalise AI responsibly.
Policy and regulation: where the state has stepped in
The UK government’s October ministerial letter to FTSE100/250 firms is notable because it shifts responsibility to corporate leadership, requires sign‑up to national defensive services, and recommends supply‑chain certification. This is a recognition that market incentives alone have not delivered sufficient resilience. The NCSC’s Annual Review functions as both an advisory document and a political statement: national security, economic stability and public safety are intertwined with corporate cyber posture. Regulatory ripples will follow: expect tighter procurement standards, enhanced supply chain audits, and possible sector‑specific resilience requirements. Boards should prepare for increased regulatory scrutiny and for the possibility of mandated minimum controls in sectors where outages create safety risks.Strengths, gaps and the path forward — a critical analysis
Notable strengths in the current approach
- The NCSC’s public framing and toolkit approach reduces friction for smaller organisations and creates common standards boards can adopt immediately. The combination of public guidance and targeted services (Early Warning, takedown, training) lowers the bar to entry for resilience measures.
- The private sector is deploying practical AI controls (Purview DSPM, Security Copilot) that make it possible to detect risky agent behaviour and to govern data flows at scale. These are real tools with measurable outcomes when integrated properly.
Material risks and blind spots
- Overreliance on AI without governance is dangerous. If an organisation adopts generative agents but fails to instrument their decisions and to apply DLP and audit trails, it trades one blind spot for another. Anthropic’s experiments make this danger concrete: models can appear to “plan,” but their internal explanations are not reliable proof of safety.
- Cost asymmetry creates a concentration risk. Larger firms can afford adaptive AI SOCs and bespoke DS&P tooling; smaller suppliers cannot. The gap amplifies systemic risk because the weak link shifts from big players to the small vendors that keep supply chains running. The ministerial letter’s supply‑chain ask is a reasonable corrective, but enforcement and capacity building will be required.
- Unverified technical claims can misdirect policy. Media and vendor narratives sometimes paint worst‑case AI scenarios as inevitable. While scenario planning must include plausible adversary advances, policy should prioritise observed capability trends and reproducible tests rather than headline speculation. Any claim about malware that “already” plans like an agent should be treated as conditional unless publicly demonstrated by multiple independent technical analyses.
Immediate checklist for boards and CNI operators
- Confirm board-level cyber training has been completed and documented.
- Ensure the organisation is signed up for NCSC Early Warning and uses the Cyber Governance Code of Practice.
- Audit identity posture: remove static admin passwords, introduce passkeys/FIDO2, and deploy PAM.
- Adopt or require DSPM/DLP controls for any deployed Copilot/agent systems.
- Rehearse an incident response playbook that includes supplier and communication channels, and run tabletop exercises with the board and communications leads.
- Validate backup isolation (immutable snapshots) and recovery target times for systems supporting safety or public services.
- Insure conservatively — check cyber insurance policies for exclusions, especially for supply‑chain and physical damage exposures.
Conclusion
The NCSC’s Annual Review is a corrective: it demands that cyber resilience move out of technical appendices and into board agendas, capital spending plans and national industrial policy. AI accelerates both threats and defences; it is not a silver bullet nor an automatic liability — its value depends on the governance that surrounds it. For Critical National Infrastructure, the calculus is straightforward: invest in identity, governance, resilient architecture, and rehearsed recovery now, because when a high‑impact incident arrives the cost of inaction will be measured in livelihoods, supply‑chain collapse, and taxpayer bailouts.The path forward is pragmatic and organisational, not purely technological. Trusted partnerships with vendors who can demonstrate audited, testable AI safety controls; cross‑sector intelligence sharing that turns indicators into collective shields; and a board that treats cyber as a strategic, measurable risk will turn the NCSC’s urgent warning into an operational advantage. The future resilience of the UK’s critical systems will be determined by the speed at which leaders convert this warning into governance, funding and disciplined execution.
Source: The AI Journal Cyber Resilience and AI Risk: Safeguarding the UK’s Critical Infrastructure in a New Threat Landscape | The AI Journal
