The cost of artificial intelligence (AI) models continues to decrease.
Token prices are falling, models are becoming more efficient, and companies can now access capabilities that were still expensive only a few years ago. At first glance, this should make AI increasingly affordable.
However, as AI agents become more widely adopted, a new challenge is emerging: the cost of the model is no longer necessarily representative of the actual cost of completing a task.
Gartner recently projected that inference costs per agentic workflow could increase more than fivefold through 2028, despite the continued decline in token prices. This reflects what Gartner describes as an “inference paradox”: as AI becomes cheaper and more capable, organizations tend to use it more extensively and for increasingly complex tasks. (gartner.com)
The question is therefore no longer simply how much one million tokens cost.
Companies increasingly need to understand how much it actually costs to complete an entire task using an AI agent.
Why Agents Cost More Than Chatbots
A traditional chatbot generally follows a relatively simple model:
User → AI model → Response
An AI agent works differently.
To complete a task, it may need to:
- Analyze the request.
- Plan the necessary steps.
- Call a tool.
- Analyze the result.
- Adjust its approach.
- Call another tool.
- Verify the result.
- Generate a final response.
A single request can therefore involve multiple model calls as well as numerous additional operations.
Recent research into agentic workloads shows that these systems behave very differently from traditional LLM applications. Agents combine inference, tool calls, data retrieval, orchestration and persistent state, creating additional compute, memory and communication requirements. (arxiv.org)
In other words, an agent does not only consume tokens. It consumes an architecture.
Context Is Also a Cost
Another frequently underestimated factor is the amount of context required by an agent.
At each step, the model may need to process:
- System instructions
- Conversation history
- Results from previous tools
- Retrieved data
- Available tool definitions
- Information required to determine the next action
As workflows become more complex, the amount of information being processed can increase significantly.
This means that an agent can quickly consume much more context than a simple chatbot interaction.
The cost is therefore not limited to the size of the final response. It can accumulate at every stage of reasoning and execution.
Recent research on agentic systems has also highlighted how context and supporting components can represent a significant part of the workload, while redundant tool calls create opportunities for optimization and caching. (arxiv)
The Most Powerful Model Is Not Always the Best Choice
Another common mistake is using the most advanced model for every task.
A model capable of solving complex reasoning problems is not necessarily required to classify an email, extract information from a document or determine whether a request should be routed to another department.
Within an agentic architecture, different tasks can require different levels of capability.
For example:
A more powerful model can be used for complex planning and decision-making.
A smaller model can handle repetitive and predictable tasks.
Deterministic automation can be used when no AI reasoning is actually required.
This approach, often referred to as model routing, allows organizations to select the appropriate model based on the complexity of each task.
Companies are already experimenting with these strategies at scale. Uber, for example, recently reported that its weekly AI-agent requests had increased 9.4× since February 2026 while keeping overall AI costs relatively stable through optimization and model selection. (axios.com)
The principle is relatively simple:
Do not use an expensive model when a cheaper one can reliably complete the task.
Automation Can Still Be More Efficient Than AI
This may seem counterintuitive at a time when companies are looking to integrate AI into almost every process.
But not every task requires an agent.
If a process always follows the same steps, traditional automation can be faster, more predictable and less expensive.
For example:
If an invoice arrives → extract the information → verify the amount → record it in the ERP → send a notification.
If the rules are clearly defined, there may be little reason to have an AI agent make a decision at every step.
On the other hand, if the process requires interpreting a request, retrieving information from several systems and dynamically determining what should happen next, an agent can provide significant value.
The goal is therefore not to replace automation with AI.
It is to determine where AI reasoning provides enough additional value to justify its cost.
The Real Cost Is Not the Token
The price charged by model providers remains an important metric, but it represents only part of the equation.
The actual cost of an AI agent can also include:
- Multiple model calls
- API and tool calls
- Data retrieval
- Infrastructure
- Context storage and management
- Monitoring
- Security
- Error handling and retries
- Human approvals
- Additional work caused by incorrect outputs
An agent that costs only a few cents per interaction may appear extremely inexpensive.
But if that interaction is repeated thousands or millions of times, or if every task requires multiple model calls and interactions with several systems, costs can increase quickly.
This is why the most useful metric may no longer simply be:
“Cost per million tokens.”
Instead, organizations may need to consider:
“Cost per successful business task.”
This distinction becomes increasingly important as companies move from small AI prototypes to large-scale deployments.
A New Discipline: AI Cost Optimization
As AI agents become more widespread, companies will need to develop new approaches to managing AI costs.
Some of these practices are relatively straightforward:
- Measure: Understand which models are being used, for which tasks, with how many calls and how much context.
- Route: Direct each task to the appropriate model instead of automatically using the most powerful one.
- Reduce context: Avoid sending unnecessary information at every stage of a workflow.
- Cache: Avoid repeating identical searches or tool calls when results can be reused.
- Limit loops: An agent that continues reasoning or calling tools without making progress can quickly generate unnecessary costs.
- Automate when possible: When a task is deterministic, a traditional workflow may be more efficient than an agent.
These optimizations are therefore not simply about finding the cheapest model.
They are about designing an architecture that uses AI where it creates real value.
The Real Question: Cost vs. Value
The continued decline in token prices is good news for businesses.
But it can also encourage much broader AI adoption.
This is what creates the current paradox: a technology can become cheaper while generating a larger overall bill because it enables companies to automate more tasks and build more complex workflows.
The objective should therefore not be to make every AI interaction as inexpensive as possible.
It should be to find the right balance between cost, performance, reliability and business value.
In a well-designed architecture, an AI agent does not necessarily replace a traditional workflow. It is introduced where it provides an additional capability: understanding a request, making a decision, adapting a process or orchestrating multiple systems.
The future of enterprise AI will therefore likely not be built around a single model capable of doing everything.
Instead, it will rely on a combination of specialized models, AI agents, deterministic automation and infrastructure capable of routing each task to the most appropriate solution.
As the cost of artificial intelligence continues to decline, the question will no longer simply be:
“How much does AI cost?”
It will increasingly become:
“How much does it cost to use AI to accomplish this task, and what value does that usage actually generate?”