DeepSeek has made headlines by slashing the price of its V4-Pro AI model by a staggering 75%, a move that many enterprise AI vendors and developers initially welcomed. However, the reality is becoming increasingly clear: lower prices don’t necessarily translate to healthier profit margins.
The main issue lies in the inference costs; while these are noticeably dropping, the agent systems are consuming tokens at an alarming rate. For over two decades, the software economy has adhered to a simple principle: infrastructure costs decrease while application capabilities expand. The expectation was that AI would follow a similar trajectory. As the Frontier model advances and token prices decline, many assumed that operational costs would diminish as well, but that assumption is now proving to be unfounded.
In typical chatbot implementations, one user query generates one model call. In contrast, the agent system transforms this into a complex chain of planning, searching, tool utilization, validation, summarization, and decision-making. While users see just one answer, vendors are billed for an extensive process – a problem that could represent a multiplier of 100x. The same user request can incur significantly more costs when processed as an agent workflow compared to chatbot responses or retrieval-augmented generation (RAG) outputs. In prolonged workflows, this escalation in costs becomes even more pronounced. Lowering the model prices helps, but it fails to resolve the fundamental issue concerning inefficient product architecture, which turns a single prompt into numerous billable operations.
The scale of the problem is culminating in the way that model providers value their partnerships with developers. OpenAI’s initiative offering every Y Combinator startup $2 million in API credits highlights the harsh reality of operating AI-native companies within the first year. This figure could have funded entire seed rounds just a few years ago when businesses managed with minimal AWS credits. For established companies integrating agents into existing product lines, the financial implications are even more daunting.
Understanding Token Amplification
In a single-turn chatbot, the ratio of user messages to model calls is typically 1:1. However, for multi-step agents involved in customer support, sales operations, finance, legal review, and engineering, the ratio can exceed 1:700. Each iteration encompasses cumulative conversation, tool outputs, and inference traces without dropping anything.
For example, consider a seemingly simple agent query: “What did our top customers ask about last week?“. This type of query often requires processing from seven different operations.
-
User prompt (~50 tokens)
-
System prompts and tool definitions (~3,000 tokens, replicated with each call)
-
Acquire context (~5,000 tokens)
-
Model Call #1 — Tool Selection (8,000 Inputs / 200 Outputs)
-
Run the tool (~4,000 tokens returned)
-
Model Call #2 — Summary (12,000 inputs / 400 outputs)
-
Model Call #3 — Follow-up decisions (12,400 inputs / 100 outputs)
This results in approximately 35,000 input tokens charged per sentence, translating to costs ranging from $0.10 to $0.40 per query for the frontier model. With an enterprise handling about 1 million queries monthly, costs can quickly escalate into six-figure expenses.
Disrupting Existing AI Business Models
The prevailing pricing structure for enterprise AI has been Seat-based SaaS: a monthly fee charged per user for agent capabilities. This model is predicated on manageable user costs.
Unfortunately, token amplification undermines this premise. For example, a power user making 50 agent calls daily under a $40/seat plan could end up incurring inference charges exceeding the subscription cost. Token amplification thus challenges traditional SaaS pricing strategies. When the costs for a power user’s daily agent usage surpass the subscription fee, vendors face detrimental gross profit margins. This challenge intensifies as customer deployments grow, leading to noticeable margin compression throughout the industry.
Recent reports are surfacing about the widening gap in functionality versus delivery within companies like Salesforce. The gap between projected capabilities and what’s actually delivered stems from the technical feasibility versus its economic viability under the planned pricing model. Unfortunately, Salesforce’s situation is not an isolated instance but part of a larger trend.
“For my team, the cost of computing far exceeds the cost of employees.” — Brian Catanzaro, Vice President of Applied Deep Learning, NVIDIA.
The underlying takeaway is not that AI is prohibitively expensive; rather, the anticipated business models embraced by many AI-native companies are increasingly untenable when faced with agent workloads.
Concrete Example
Imagine an enterprise software provider that charges $40 per user each month for an AI-driven support assistant. Traditionally, the inference costs incurred by conventional chatbots were negligible, enabling healthy profit margins.
However, replacing the chatbot with a comprehensive agent workflow—capable of ticket investigations, querying internal systems, drafting responses, validating content, and escalating issues—can lead to exponential increases in inference use. A heavy user performing 50-100 agent requests daily can make what was once an almost negligible cost into a monumental operating expenditure.
This unusual dynamic reveals that customers deriving significant value from your product may also be the ones incurring the highest inference expenses. This raises concerns as top users can inadvertently undermine profit margins. Consequently, there’s an increasing recognition that agent adoption and profit margins are no longer inherently aligned in the realm of enterprise software.
Agent Orchestration as a Competitive Advantage
The technical strategies to address these challenges are recognized but critical for survival:
-
Cost-aware Routing: Implementing a small classifier model to determine which processing layer (Haiku, Sonnet, Opus) manages each query can effectively reduce inference costs by around 60% without sacrificing quality.
-
Instant Cash: Anthropic, OpenAI, and Google currently provide substantial discounts (75-90%) on cached prefixes.
-
Context Discipline: Techniques that truncate tool outputs, clean up inference traces, and limit tool depth help prevent agents from going down costly rabbit holes.
-
Speculative Decoding: Particularly for self-hosted setups, this method ensures 2-3x improved throughput on the same GPU.
“Organizations leveraging orchestration-driven governance report substantially tighter productivity gains. A well-implemented orchestration layer can enhance productivity up to six times more than compliance-based methods.” IBM
Firms successfully cultivating this orchestration layer are evolving to resemble more sophisticated operational models akin to financial trading systems, where every routing decision carries a cost, and every pathway has its own profit and loss metrics.
Recommended Actions for Corporate Leaders
Decisions made today will determine which companies maintain margins in the next 24 months and which falter.
-
Prioritize Inference Cost Tracking: Monitor inference costs in a manner similar to how cloud costs have been tracked since the 2010s: segmented by feature, tenant, and query type.
-
Establish an Inference Budget: Implement a cost cap per 1,000 queries for each feature and issue warnings upon overspending; engineering departments often don’t enforce these limits independently.
-
Recognize Your Router as Core Infrastructure: Treat it like a crucial component rather than a mere optimization.
-
Conduct Quarterly Prompt Audits: A continuously growing system prompt of 4,000 tokens can translate into significant hidden costs over time. Teams should regularly review their production prompts thoroughly.
-
Negotiate Volume Commitments Early: Many frontier model vendors currently offer deep discounts for upfront Reserved Instance commitments; the list price represents the highest potential cost to a company.
The Next 24 Months
The fundamental change influencing agent AI isn’t that AI costs are soaring. As highlighted by DeepSeek’s price reduction, unit costs for frontier inference have dropped roughly 3x annually without signs of slowing.
What’s the underlying shift? The amplification rates are outpacing price reductions. A 75% decrease in token cost won’t alleviate the burdens placed on a company whose agents are processing upwards of 700 times more tokens than predicted with user-based pricing models. For the first time since the dawn of cloud infrastructure, architecture-related decisions are critically influencing financial strategies in real-time.
As a result, rapid redesigns will have significant financial implications. A poorly optimized agent loop can translate directly into financial losses for companies.
In the upcoming 24 months, the firms that will thrive in the evolving AI infrastructure landscape won’t merely be those running the most affordable models; they will be the ones that strategically manage their agent costs and possess deep insights into operational expenditures.
These challenges linked to 100x operational costs are arriving sooner than any temporary price cuts can conceal.
Maitreyi Chatterjee is a senior software engineer at a leading technology company.
Devansh Agarwal works as an ML engineer at a large technology company.
Welcome to the VentureBeat community!
This guest posting program enables tech leaders to contribute insights and provide an unbiased, rights-free comprehensive analysis on AI, data infrastructure, cybersecurity, and other transformative technologies shaping the enterprise landscape.
For more information, check out our guest posting guidelines if you are interested in contributing your own article!
Source: venturebeat.com