Conventional chatbots offer a simple cost structure: a user sends in a prompt, the model generates a response, and the company pays for the tokens involved. Agentic AI systems are something else. Give one an objective, and it might search the web, call an API, write code, consult another model, review its work, and then try, try again.
For the end user, that means higher costs that are hard to predict or forecast. The tokens might not be expensive, but when a single task branches out into model calls, tool calls, and larger context windows, those tokens pile up. Agents communicate with each other, and external tools add their own charges. As the query is pursued, it becomes more expensive.
Is there a smarter way to do things? Certainly. You can ask questions to stay on budget. Before your next review, ask these five questions to identify where the money is going.
1. What format does your vendor actually return?
While the format of an API response might seem like an engineering detail, it can become a cost issue when dealing with AI agents. Many APIs default to JavaScript Object Notation (JSON), a lightweight text format designed for software to parse. But LLMs don’t need every bracket, field name or piece of metadata in a long JSON response.
Some vendors offer alternative formats, such as Markdown, that can reduce token costs. SerpApi, for example, offers Markdown output tailored specifically for agentic workflows. In a head-to-head comparison, the company found that a search response requiring 24,723 tokens in JSON needed only 6,435 in Markdown—a reduction of roughly three-quarters. SerpApi has also rolled out a JSON Restrictor feature that allows developers to trim responses.
Have a look at your vendor’s API documentation and see if it offers an alternative to JSON. This simple question could translate into immediate savings.
2. How much of what you’re paying for actually gets used?
A user might get a detail-rich API response, but do they really need it all? Data might be returned that ultimately goes unused. Maybe it’s time to ask yourself: Is it necessary to pay for something you’ll never use?
Take a representative API call and trace it through the entire workflow. Measure the size of the response, then determine which information the model actually uses in the next step. The difference gives you a practical estimate of your payload waste, grounded in a real use case.
3. Can you filter the response down to only what you need?
Even with a more efficient format, you might still pay for information your agent doesn’t need, and a little unnecessary information, repeated often enough, can become a real expense.
Some vendors allow developers to specify exactly which fields they want returned. These options might be called “fields,” “include,” “restrict,” or something similar. The question is simple: Can you tell the API what you actually need, rather than accepting everything it has to offer?
4. What does a retry actually cost you?
Here’s where things can get eye-rollingly expensive. Agents are supposed to be autonomous. That means that if something goes wrong, they can, and will, try again. All of these attempts cost more money. Often, agents start over again, which means you end up paying for the entire chain multiple times over.
To cut down on retry costs, have a look at your logs and find out how many retries your agents made over the past week. Then check the average context size for those calls. That will give you a better idea of what retries are actually costing.
5. How far off is your estimate from reality?
AI budgets are often built around estimates, but agentic systems don’t always match those parameters. Therefore, the issue isn’t just the number of hidden costs, it’s that one’s own budget was flawed to begin with.
Overcoming this issue requires some research. Get a better handle on what your query will cost. Take a genuine workflow and measure its token consumption. Then tweak it and run it again. This will help you better understand what your query might really cost in the end.
The final takeaway
It’s convenient to think of AI agent use as a token game, but avoiding ballooning costs takes more time, attention, and a mindset shift.
While it can seem daunting, companies are never fully at the mercy of AI. With the right questions, they can get the answers they need at a reasonable cost.
