Efficiency-Driven Cost Model: Maximizing ROI through Successful Task Completion

AI/ML
Last Updated: September 24, 2026
by Patel Akash

Most AI agent budgets are based on the provider’s price per million tokens, but this number is often inaccurate by a significant margin. The token price only accounts for the cost of one model call and does not consider factors such as the number of calls per task, the amount of context resent on each call, the failure rate, or the cost of human intervention when the agent fails.

To better understand what drives the cost of AI agents, we developed a cost model for three common agent workloads. The results showed that the true unit cost is often understated by a factor of 19 to 72 times compared to the per-call price. In two out of the three workloads analyzed, model tokens accounted for less than a quarter of the total cost. This article delves into the model, the mathematics behind it, and the key factors that impact costs, allowing you to budget for an agent before it goes into production.

Key takeaways:

– Measure cost per successful task, not cost per token, as it provides a more accurate comparison with current work costs.
– Agents resend their context on every step, resulting in a growth in billed input tokens with the square of the number of steps.
– Success rate is a significant factor in cost reduction. Increasing the agent’s resolution rate by 10% led to a 37% decrease in cost per task.
– Caching plays a crucial role in reducing costs for long-running agents, with a 73% cost increase per successful task without caching.

Why price per token is the wrong starting number:

Providers’ price sheets only address the cost of sending a specific number of tokens to a model and receiving a response. However, an agent consists of multiple calls, each with its own set of tasks and context. This context accumulates over time, and failures incur additional costs. Additionally, output tokens are often priced higher than input tokens, adding to the overall cost.

The cost-per-successful-task formula:

The recommended metric for calculating cost is the cost per successful task, which includes model spend on all attempts, human review and rescue costs, and allocated platform costs, divided by the number of tasks completed correctly. This formula accounts for failed attempts, automatic retries, and secondary models used in the process.

How agent token usage actually grows:

As an agent progresses through tasks, the number of input tokens billed grows exponentially with the square of the step count. Caching is a critical countermeasure to reduce costs, with providers offering discounts for cached input reads.

Three worked models:

We analyzed three common agent deployments: a real-time support agent, a batch document processing pipeline, and a long-running operations agent. Each model had different token prices, steps per task, success rates, and human review costs, affecting the overall cost per successful task.

Cost per call vs. cost per successful task:

The cost per successful task is what the business actually pays, and it can be significantly higher than the per-call estimate. The ratio between the two grows with loop depth, highlighting the importance of accurate budgeting for long-running agents.

Which levers actually move the bill:

Success rate and human costs have the most significant impact on costs, followed by architecture choices such as caching, batching, and loop depth. Token price has a moderate effect on costs and may not be the most significant cost-saving measure.

How to reduce cost per successful task:

Optimizing token spend, measuring success rates accurately, and making strategic changes to increase success rates can help reduce costs. Implementing caching strategies, routing tasks efficiently, and batching non-urgent tasks can also contribute to cost reduction.

Budget controls that prevent runaway spend:

Implementing per-task limits, per-customer or per-tenant budgets, anomaly alerts, and a kill switch can help prevent an AI agent from running up costs. These controls should be integrated into the orchestration layer to ensure effective cost management.

Comparing against the human baseline:

To assess the cost-effectiveness of an AI agent, it’s essential to compare the cost per successful task to the cost of achieving the same outcome with human intervention. Building a baseline that accounts for human intervention and errors can provide a clear comparison between AI and human costs.

What to measure from day one:

Monitoring task volume, success rates, escalation rates, token usage, model spend, human review costs, and cost per successful task on a weekly basis can help track the performance and cost-effectiveness of an AI agent. These metrics can guide decision-making for prompt changes, model updates, and tool improvements.

In conclusion, understanding the true cost drivers of AI agents and implementing strategic measures to reduce costs are essential for effective budgeting and cost management. By focusing on key factors such as success rate, human costs, and caching strategies, businesses can optimize the cost-effectiveness of their AI agents and achieve significant cost savings over time.