LLM Cost

Calculator guide

How much does a long-running agent cost if you send the whole chat every turn?

If every agent turn re-sends the whole chat, input tokens do not grow linearly — they grow like a triangle. This guide walks the math, runs one labeled multi-session example, then sends you to the cost calculator with your own numbers.

What “full context every turn” means

A common agent loop keeps the system prompt, tools, and the entire transcript in the request. Turn 1 is cheap. Turn 20 re-pays for turns 1–19 again. That is useful for continuity — and expensive when sessions are long or frequent.

Why cost grows faster than “tokens × turns”

For a fixed prefix P, user message size U, assistant reply size A, and T turns, total input tokens are approximately:

Σ I_t = T·P + T·U + (U+A)·T(T−1)/2

Output is closer to linear (T·A). The triangle term is why “we only added a few turns” can still blow the input bill.

Worked example (illustrative assumptions)

Labeled scenario — not customer data: 200 sessions/day × 20 turns, with P=2500, U=150, A=300.

  • Per session ≈ 138.5k input / 6k output tokens
  • Monthly (30 days) ≈ 831M input / 36M output
  • Same workload retrieving R=1500 tokens per turn instead of dumping history ≈ 498M input (same output) — derived from the same P/U assumptions plus R, not a hard-coded monthly total
Cumulative input tokens by turn for the worked example
TurnCumulative input tokensvs turn 1
12,6501.0×
517,7506.7×
1046,75017.6×
20138,50052.3×
Illustrative monthly cost for the worked example on two catalog models
Model (catalog snapshot)Full-context / moRetrieved-context / mo
GPT-4o mini
$0.15/M in · $0.6/M out
$146$96.3
Claude Sonnet 4
$3/M in · $15/M out
$3,033$2,034

Dollar columns use a catalog snapshot as of 2026-09-23. Prices move — re-check on the calculator before you publish a budget.

Levers that usually move the bill

  1. Trim or summarize older turns instead of re-sending them verbatim.
  2. Use provider prompt caching when the prefix is stable.
  3. Pick a cheaper input $/M model when quality still holds — compare on /compare.
  4. Retrieve only what the next turn needs instead of dumping the transcript.
  5. Scan /cheapest-llm-api and /models when input-heavy workloads dominate.

Soft context: interaction-scoped memory (for example the Mem0 paper on Memory Papers) is one way teams avoid full-transcript dumps — this guide stays on cost math.

Run your numbers

  1. Open /calculator and enter your input/output tokens per session (or per day).
  2. Compare two models on /compare when you are choosing a cheaper input tier.
  3. Use /cheapest-llm-api for a ranked shortlist, then confirm on the model pages under /models.
Open cost calculator →Compare models

FAQ

Because turn N usually re-sends turns 1…N−1. Input grows like a triangle: roughly T·P + T·U + (U+A)·T(T−1)/2 for fixed prefix P, user message U, and assistant reply A across T turns — not T·(P+U+A).