The relentless pursuit of ever-larger generative AI models has become a defining trend in the technology industry, dominating the ambitions of major players and enterprise buyers alike. However, mounting evidence suggests that this obsession with size is delivering diminishing returns—and may even pose existential threats to AI projects’ reliability and economic viability. Drawing from the perspectives of industry experts and engineers, it’s clear that smaller, more focused AI models may hold the key to unlocking sustainable generative AI adoption within the enterprise.
Over the past two years, the leading names in AI—OpenAI, Microsoft, Google, Amazon, Anthropic, and Perplexity—have made stunning advances in model scale. Each new iteration touts ever-larger parameter counts, tantalizing enterprises with the promise of near-magical capabilities: natural conversation, agent-driven automation, and reasoning prowess matching (or even besting) humans. According to public communications and technical reports, OpenAI’s GPT-4 Turbo boasts trillions of parameters, while Google’s Gemini models and Anthropic’s Claude line are built upon similarly massive deep neural architectures.
Proponents of this “bigger is better” mantra argue that larger models capture richer world knowledge, offer more flexibility, and can solve more complex problems. Indeed, for tasks such as open-ended language generation and broad conversational interfaces, model scale does appear to strongly influence performance.
But there is mounting concern—backed by both empirical data and operational experience—that this scale-centric strategy introduces a cascade of technical and economic risk as soon as enterprises attempt to deploy generative AI at production scale.
Kanwat observed that even models achieving a high per-step reliability rate—let’s say an optimistic 95%—quickly see their chance of completing multi-step, autonomous agent workflows plummet as task complexity grows. For a five-step process, the end-to-end success rate is already down to around 77%. Double the length to ten steps and the success probability falls to 59%. At 20 steps—common in enterprise processes—the chance of success drops below 36%.
Critically, Kanwat noted that production-grade enterprise systems require “99.9%+ reliability.” Even reaching an unprecedented 99% step reliability would leave a 20-step process achieving only 82% overall reliability—far below operational thresholds. No matter how sophisticated the prompt engineering, no matter how advanced the model, the underlying mathematics render highly autonomous, multi-step AI agents unsustainable for mission-critical use. This, he warned, is not just a technical challenge, but a fundamental limitation.
Multiple voices in the AI research and analyst community have echoed Kanwat’s analysis. Jason Andersen, principal analyst at Moor Insights & Strategy, emphasized that broad, generalized models promise much—yet consistently fail to meet reliability expectations when scaled to real workflows. The more steps and context introduced, the harder it becomes to keep error rates from spiraling out of control.
Kanwat highlighted a particularly thorny issue with today’s leading large language models: the quadratic escalation of costs with conversation length. In practical terms, the token expenses for carrying forward prior context and history in a typical 100-turn dialogue session can reach $50–100, even before factoring in the infrastructure and integration overhead. As usage scales to thousands of employees or customers, these costs quickly become unsustainable.
The fate of many VC-backed autonomous agent startups hangs in the balance here. While flashy five-step demo workflows may wow investors, enterprise buyers eventually demand 20+ step integrations that expose the reliability drop-off and trigger cost explosions. Kanwat predicted that these startups will soon “hit the economics wall,” with burn rates spiking as they attempt to repair what are, in effect, mathematically unsolvable reliability problems.
Andersen confirmed that pricing models for major LLM providers are already trending upwards, particularly as models are integrated into more context-heavy, multi-agent applications. While some providers mask these costs early on, there’s industry consensus that enterprises should brace for significant increases—especially if usage is left unchecked.
Himanshu Tyagi, co-founder of AI vendor Sentient, outlined a compelling middle ground: “There’s a trade-off between deep reasoning and streamlined reliability. Both should coexist, not compete.” Over-sized, generalist AI systems optimize for lock-in and infrastructure scale, eating up budgets but rarely delivering deterministic value. Fine-tuned, leaner models—perhaps even multiple models optimized for individual subsystems—can deliver superior performance in production, with sharply reduced error rates and resource requirements.
Robin Brattel, CEO of Lab 1, noted that AI agents focused on discrete, well-bounded tasks are less prone to compounding errors and far better able to meet real-world enterprise requirements. Smaller models, by virtue of being tightly scoped, allow for greater control and validation, greatly increasing their likelihood of success in complex operational environments.
Chester Wisniewski, director of global field CISO at Sophos, added his support, arguing that “if you hypertrain a neural network to do one thing, it will do it better, faster, and cheaper. If you train a very small model, it is far more efficient.” The drawback, Wisniewski conceded, is that it places a heavier demand on enterprise IT and data science teams, who must now manage, train, and deploy numerous specialized models rather than simply onboarding the latest all-purpose behemoth.
But many industry practitioners are urging CIOs to move past the allure of all-knowing, AI-powered copilots and instead reconsider their goals: Should AI act as a pilot—making decisions unsupervised—or as a navigator, offering recommendations and guidance while keeping the ultimate responsibility with humans?
Jason Andersen’s analogy is instructive: Expecting a generalized AI agent to be dropped into a legacy enterprise system and figure everything out is akin to hiring a new employee and refusing to train them—then blaming them when inevitable misunderstandings occur. The “plug-and-play” hope masks the deep complexity within most enterprise environments: legacy integrations, partial failure modes, authentication flows that change overnight, varying compliance regimes, and a patchwork of business logic.
Kanwat argued that many enterprises have misapplied generative AI by trying to layer all-purpose agents on top of messy legacy systems. The result is stagnation in adoption, decelerating project delivery, and escalating costs. The solution, he suggested, is to build targeted AI-driven tools that solve specific problems well—rather than attempting an all-knowing, conversational interface for everything.
By focusing aggressively on well-bounded domains and tightly limiting the range of allowed queries, Capital One has sidestepped many of the issues that plague more diffuse AI deployments. Their results show that carefully curated, problem-first generative AI can deliver enterprise value—without being eaten alive by error rates or runaway costs.
Industry experts recommend weighing the following factors when selecting model scale and architecture:
Moreover, the infrastructure ecosystems being developed for model orchestration, dataset curation, and multi-model collaboration are laying the groundwork for ever more sophisticated hybrid systems. Forward-looking organizations can harness the strengths of both paradigms—leveraging large models for broad knowledge and creativity, while confidently deploying small, domain-trained models for mission-critical, high-precision workflows.
As the breathless headlines about trillion-parameter models continue to circulate, enterprises would be well served by taking a sober, evidence-based view of their true requirements. The future of AI in business may not be built on the largest model, but on the wisest application of the right-sized models, working together with clarity of purpose and measurable ROI.
Source: theregister.com AI industry's size obsession is killing ROI, engineer argues
Over the past two years, the leading names in AI—OpenAI, Microsoft, Google, Amazon, Anthropic, and Perplexity—have made stunning advances in model scale. Each new iteration touts ever-larger parameter counts, tantalizing enterprises with the promise of near-magical capabilities: natural conversation, agent-driven automation, and reasoning prowess matching (or even besting) humans. According to public communications and technical reports, OpenAI’s GPT-4 Turbo boasts trillions of parameters, while Google’s Gemini models and Anthropic’s Claude line are built upon similarly massive deep neural architectures.Proponents of this “bigger is better” mantra argue that larger models capture richer world knowledge, offer more flexibility, and can solve more complex problems. Indeed, for tasks such as open-ended language generation and broad conversational interfaces, model scale does appear to strongly influence performance.
But there is mounting concern—backed by both empirical data and operational experience—that this scale-centric strategy introduces a cascade of technical and economic risk as soon as enterprises attempt to deploy generative AI at production scale.
Reliability Falls Off a Cliff
Utkarsh Kanwat, an AI engineer with Australia’s ANZ Bank, brought data-driven clarity to the problem in a widely read blog post. He challenged the industry’s prevailing dogma by laying out the uncomfortable mathematics at the heart of agent-driven AI workflows.Kanwat observed that even models achieving a high per-step reliability rate—let’s say an optimistic 95%—quickly see their chance of completing multi-step, autonomous agent workflows plummet as task complexity grows. For a five-step process, the end-to-end success rate is already down to around 77%. Double the length to ten steps and the success probability falls to 59%. At 20 steps—common in enterprise processes—the chance of success drops below 36%.
Critically, Kanwat noted that production-grade enterprise systems require “99.9%+ reliability.” Even reaching an unprecedented 99% step reliability would leave a 20-step process achieving only 82% overall reliability—far below operational thresholds. No matter how sophisticated the prompt engineering, no matter how advanced the model, the underlying mathematics render highly autonomous, multi-step AI agents unsustainable for mission-critical use. This, he warned, is not just a technical challenge, but a fundamental limitation.
Multiple voices in the AI research and analyst community have echoed Kanwat’s analysis. Jason Andersen, principal analyst at Moor Insights & Strategy, emphasized that broad, generalized models promise much—yet consistently fail to meet reliability expectations when scaled to real workflows. The more steps and context introduced, the harder it becomes to keep error rates from spiraling out of control.
The Intractable Economics of Large Models
The reliability problem feeds directly into project economics. Enterprise AI teams are discovering that, as models get larger and more context turns are required, costs escalate rapidly—often in a way that outpaces any return on investment.Kanwat highlighted a particularly thorny issue with today’s leading large language models: the quadratic escalation of costs with conversation length. In practical terms, the token expenses for carrying forward prior context and history in a typical 100-turn dialogue session can reach $50–100, even before factoring in the infrastructure and integration overhead. As usage scales to thousands of employees or customers, these costs quickly become unsustainable.
The fate of many VC-backed autonomous agent startups hangs in the balance here. While flashy five-step demo workflows may wow investors, enterprise buyers eventually demand 20+ step integrations that expose the reliability drop-off and trigger cost explosions. Kanwat predicted that these startups will soon “hit the economics wall,” with burn rates spiking as they attempt to repair what are, in effect, mathematically unsolvable reliability problems.
Andersen confirmed that pricing models for major LLM providers are already trending upwards, particularly as models are integrated into more context-heavy, multi-agent applications. While some providers mask these costs early on, there’s industry consensus that enterprises should brace for significant increases—especially if usage is left unchecked.
Smaller and Smarter: The Case for Domain-Specific Models
The logical alternative to this scale-obsessed approach is to deploy smaller, context-specific models tailored to precise enterprise needs. While this can sound regressive to those dazzled by AI hype, technical experts increasingly argue it is the only sustainable path forward.Himanshu Tyagi, co-founder of AI vendor Sentient, outlined a compelling middle ground: “There’s a trade-off between deep reasoning and streamlined reliability. Both should coexist, not compete.” Over-sized, generalist AI systems optimize for lock-in and infrastructure scale, eating up budgets but rarely delivering deterministic value. Fine-tuned, leaner models—perhaps even multiple models optimized for individual subsystems—can deliver superior performance in production, with sharply reduced error rates and resource requirements.
Robin Brattel, CEO of Lab 1, noted that AI agents focused on discrete, well-bounded tasks are less prone to compounding errors and far better able to meet real-world enterprise requirements. Smaller models, by virtue of being tightly scoped, allow for greater control and validation, greatly increasing their likelihood of success in complex operational environments.
Chester Wisniewski, director of global field CISO at Sophos, added his support, arguing that “if you hypertrain a neural network to do one thing, it will do it better, faster, and cheaper. If you train a very small model, it is far more efficient.” The drawback, Wisniewski conceded, is that it places a heavier demand on enterprise IT and data science teams, who must now manage, train, and deploy numerous specialized models rather than simply onboarding the latest all-purpose behemoth.
The Myth of “Plug-and-Play” AI in the Enterprise
The pressures driving the scale obsession are not purely technical; enterprise culture and vendor marketing also play a major role. With big-name vendors hailing massive models as silver bullets and the “AI as magic” narrative taking root at the executive level, there is tremendous institutional inertia behind the biggest possible model.But many industry practitioners are urging CIOs to move past the allure of all-knowing, AI-powered copilots and instead reconsider their goals: Should AI act as a pilot—making decisions unsupervised—or as a navigator, offering recommendations and guidance while keeping the ultimate responsibility with humans?
Jason Andersen’s analogy is instructive: Expecting a generalized AI agent to be dropped into a legacy enterprise system and figure everything out is akin to hiring a new employee and refusing to train them—then blaming them when inevitable misunderstandings occur. The “plug-and-play” hope masks the deep complexity within most enterprise environments: legacy integrations, partial failure modes, authentication flows that change overnight, varying compliance regimes, and a patchwork of business logic.
Kanwat argued that many enterprises have misapplied generative AI by trying to layer all-purpose agents on top of messy legacy systems. The result is stagnation in adoption, decelerating project delivery, and escalating costs. The solution, he suggested, is to build targeted AI-driven tools that solve specific problems well—rather than attempting an all-knowing, conversational interface for everything.
Case Study: Capital One’s Pragmatic Approach
Not all large enterprises have fallen headlong into the large model trap. Kanwat referenced Capital One’s internal GenAI initiatives, which notably restrict AI’s purview to strictly internal data sets, and constrain interaction to what is already known and validated within their enterprise database.By focusing aggressively on well-bounded domains and tightly limiting the range of allowed queries, Capital One has sidestepped many of the issues that plague more diffuse AI deployments. Their results show that carefully curated, problem-first generative AI can deliver enterprise value—without being eaten alive by error rates or runaway costs.
Assessing AI Model Scope: Key Criteria for IT Leaders
So, how can enterprise technology leaders make smart, defensible choices on AI adoption, amid relentless marketing pressure and ambiguous returns?Industry experts recommend weighing the following factors when selecting model scale and architecture:
- Precision Requirements: For tasks with “low precision requirements” (e.g., creative illustration or birthday card poetry), approximate correctness is acceptable. For processes tightly coupled to data integrity or compliance, high precision is mandatory.
- Risk Profile: Some tasks are inherently low-risk (generating marketing copy); others are high-stakes (autonomous driving logic, fraud detection). The higher the risk, the tighter the scope, controls, and oversight required.
- Integration Difficulty: Agent-based workflows in real-world enterprises must grapple with dirty data, inconsistent APIs, and shifting business logic. Small, modular models are more easily validated and updated when system dynamics change.
- Operational Cost Structure: Monitor not just inference rates, but also context scaling, historical session depth, and long-tail interaction costs. Seek transparency from vendors about pricing algorithms, especially for multi-turn or agentic use cases.
- Security and Privacy: Smaller, domain-specific models can better respect data sovereignty, privacy requirements, and compliance boundaries by restricting knowledge and logic to approved data sets.
The Hype Trap: Risks Lurking Ahead
The prevailing “bigger is better” hype is creating a hazardous environment—both for enterprises betting large on vendor promises and for the AI ecosystem at large. Without urgent course correction, several risks loom:- Economic Collapse of Autonomous Agent Startups: As noted by Kanwat, current venture-backed agent companies risk hitting an economics wall as soon as production usage expands.
- Enterprise Disillusionment: High failure rates and spiraling costs may prompt enterprise buyers to hit pause on innovation or retreat from AI investment, dampening industry momentum.
- Vendor Lock-in and Reduced Innovation: Large model deployments often tie organizations closely to big tech providers, eroding negotiating leverage and reducing flexibility to shift approaches as needs evolve.
- Security and Governance Nightmares: The opacity and unpredictable behavior of large models make them more vulnerable to adversarial attacks, data leakage, and compliance failures than smaller, more auditable systems.
Strengths and Untapped Potential Amidst the Turbulence
Despite these challenges, it should be underscored that large foundation models continue to expand what is possible in AI, particularly with respect to breadth of generalization and creative generation. Their value as research accelerators, idea generators, and scalable tools for low-stakes domains is real and growing.Moreover, the infrastructure ecosystems being developed for model orchestration, dataset curation, and multi-model collaboration are laying the groundwork for ever more sophisticated hybrid systems. Forward-looking organizations can harness the strengths of both paradigms—leveraging large models for broad knowledge and creativity, while confidently deploying small, domain-trained models for mission-critical, high-precision workflows.
Charting a Path Forward
As the generative AI landscape matures, CIOs, data scientists, and IT decision-makers must resist the gravitational pull of “scale at all costs.” Instead, the industry’s leaders and practitioners should:- Focus on problem-first AI adoption: Begin with real business needs and measurable outcomes, rather than a technology-first mindset.
- Prioritize control, transparency, and reliability: Small, scope-constrained models enable more reliable results, faster regulatory audits, and lower operational risk.
- Embrace modular, composable AI architectures: Rather than all-knowing agents, build federated systems in which specialized models collaborate, orchestrated by clear business logic and oversight.
- Invest in team skills and model development: Building smaller, fit-for-purpose models requires a new wave of AI talent and data engineering fluency within enterprise IT groups.
- Scrutinize vendor promises and demand evidence: Avoid being swept up in marketing cycles that obscure real costs and technical constraints; look for vendors that provide detailed performance data, transparent pricing, and strong support for integration into heterogeneous enterprise environments.
Conclusion: It’s Not About “Bigger”—It’s About “Better”
The generative AI revolution is just getting started, but it is already clear that sustainable, high-impact enterprise adoption will not be won by sheer scale. As the lessons from leading experts and early enterprise deployments make clear, precision, reliability, and economic sanity favor smaller, more controllable models—ones that are carefully matched to actual organizational needs.As the breathless headlines about trillion-parameter models continue to circulate, enterprises would be well served by taking a sober, evidence-based view of their true requirements. The future of AI in business may not be built on the largest model, but on the wisest application of the right-sized models, working together with clarity of purpose and measurable ROI.
Source: theregister.com AI industry's size obsession is killing ROI, engineer argues