AI cost per conversation: model, timing and a monthly cap

The AI cost of a conversation comes down to four decisions: which model handles each task, how much context gets reused, which hour or queue the call runs in, and what cap holds the monthly spend. The cheapest model that passes your own tests does most of the work. Caching and off-peak pricing cut the rest. The cap stops one client or one bug from pushing the bill past what was agreed.

Where the cost of a conversation comes from

Model APIs charge per token, the smallest unit of text a model reads. DeepSeek’s pricing page states the math plainly: the expense is the number of tokens times the price. Input and output are priced separately. In Anthropic’s table, an output token on Claude Haiku 4.5 costs five times an input token.

In a conversation, input grows with every turn. Each new call usually sends the system instructions and the history again, as Anthropic’s caching documentation describes. That is why the same question costs more on turn ten than on turn one.

Scope comes before the model

NN/g’s article on scope in generative AI features asks four questions before anyone designs an AI feature:

What user problem does the feature solve?
How does solving it help the business?
Is AI really the best solution?
How complete, clean and structured is the data going into the model?

The third question is also a cost question. A step with a fixed rule, like sending a message at a scheduled time, can run without calling a model at all.

The same article shows that a narrow feature restricts the type and size of input it accepts. For users, that makes the purpose clearer. For the bill, it means smaller requests and similar tasks, which tend to fit a smaller model.

One model per task

Anthropic’s guide on choosing a model describes two starting points. In the first, you start with the fastest, cheapest model, test it and move up only when capability falls short. In the second, you start with the most capable model, tune the prompts and then try to move down. The first suits simple, high-volume tasks, and the second suits complex reasoning.

In a support conversation, tasks don’t weigh the same. Classifying the intent of a message, pulling out a time slot and summarizing the day are different jobs from answering an objection. The same page suggests combining models: a cheaper model does the bulk of the work and hands the hard call to a stronger one, so most tokens are billed at the lower rate.

Tests with real cases decide the switch. The guide asks for a set of cases from your own use, with your prompts and your data, comparing accuracy, quality and edge cases. It also notes that tuning the effort parameter, on models that have one, often pays off more than switching models.

Cache the part that repeats

System instructions and the start of the history repeat on every call in a conversation. Anthropic’s pricing table bills reading that part from the cache at 10% of the base input price, or less on some models. Writing it to a five-minute cache costs 1.25 times the base price, and a one-hour cache costs double. By the page’s own math, the five-minute cache pays for itself on the first read.

The cache stores the prefix of the request. In practice, keep the fixed parts first, like instructions and business context, and put whatever changes at the end.

Timing and queues: what can wait

Three mechanisms trade waiting or timing for price:

Batch. OpenAI’s Batch API costs 50% less and completes each batch within 24 hours. Anthropic’s table gives the same 50% discount for batch processing.
Flexible queue. OpenAI’s Flex processing charges batch rates on regular calls, in exchange for slower responses and occasional resource unavailability. The docs point it at low-priority and asynchronous work.
Off-peak. DeepSeek charges half price outside peak hours. Peak runs from 01:00 to 04:00 and from 06:00 to 10:00 UTC on weekdays. In Brasília time, that is 22:00 to 01:00 and 03:00 to 07:00.

The math depends on the time zone. For a business that works Brasília office hours, the whole workday falls outside DeepSeek’s peak. Prices can change, and the page itself recommends checking them regularly.

The real limit is the patience of the person on the other end. Jakob Nielsen’s piece on the three response time limits puts 10 seconds as the ceiling for keeping attention on the dialogue. Past that, people turn to something else and need to know when the answer will arrive. A customer message can’t wait for a 24-hour batch. The daily summary, the weekly spreadsheet and a second pass over old conversations can. How to show the wait on screen is covered in the post on loading and empty states.

Monthly cap: the provider’s limit and yours

The provider already ships a limit. At Anthropic, each usage tier carries a monthly spend cap, and the organization can set a lower one. You can also set limits per workspace, which separates projects inside one account. When the tier cap is reached, the API pauses until the first day of the next month and returns a 429 error. When it is a limit you set, the error is a 400 that says when access resumes.

That limit protects the account, but it arrives late for the product. It cuts everything at once and doesn’t know which client spent the money. A per-client cap inside the application handles both: it warns before the budget runs out and blocks only whoever went past the agreement.

The block is also a screen. NN/g’s guidelines for site AI chatbots recommend one assistant that identifies itself, handles what it can and escalates to a human when it can’t. An assistant that hit its cap falls into that case, and whoever sent the message needs a path to someone on the team.

How it turned out at Vexor

In my product, Vexor, every WhatsApp conversation goes through an AI assistant, and the cost per conversation stayed in the cents with three decisions. The model is picked per task. The provider’s lowest-price window is used when the timing allows. And each client has a monthly cap with a warning and an automatic block.

I measured about R$ 0.015 (Brazilian reais) per conversation on the mid-tier model. On the lowest-price model, off-peak, it came in under one centavo, a hundredth of a real.

Of the three decisions, the cap matters most to the client. The warning comes first, and the automatic block keeps the spend from going past what was agreed. Without it, the monthly price would depend on conversation volume, which changes from month to month.

Where the sources disagree

Where to start. Anthropic’s page offers the path of starting with the cheap model and moving up only when capability falls short. The same page says most workloads start with Claude Opus 5.5, one of the most capable models in the lineup (Anthropic).
Wait or not. OpenAI’s batch promises completion within 24 hours. Flex charges the same rate without a batch, but warns that resources may be unavailable. DeepSeek’s discount asks for no waiting at all, only the right hour.
Narrow or broad. NN/g shows that narrow features are easier to understand and adopt. Anthropic’s docs recommend the most capable models precisely for long, open-ended work, like agent sessions that run for hours.

The common mistake

The mistake is picking the model from the price table and the cap from the monthly statement. A model too weak for the task gets it wrong, and the conversation comes back as rework. A cap that lives only at the provider cuts everyone off at once, with no warning, in the middle of the month. Before launch, build the test from real conversations, separate what can wait from what needs an answer now, and set a per-client cap that warns before it blocks.