Shorter Prompts Are Making Your AI…

1. Key Themes

Optimizing per-call token count is the wrong metric — the job, not the call, is the unit of cost

GitHub's own experiment shows that shrinking individual tool outputs can increase total cost because the agent has to spend extra turns recovering lost information.

"On average the task consumed more tokens and took longer. Completion rates held steady. So each call got cheaper, but the whole task got more expensive." "A response 200 tokens shorter that triggers two recovery turns is a loss. A response that runs longer but lets the agent finish in one pass is a win."

Agentic AI cost structure is fundamentally different — and dangerously non-obvious — versus chatbot usage

The multi-step, tool-calling nature of agents creates compounding, quadratic cost growth that isn't intuitive to those modeling budgets off chatbot usage.

"The attention cost scales with the square of sequence length, so a context that doubles does not cost twice as much." "Agentic workloads land somewhere around a thousand times the token consumption of ordinary prompting."

Enterprises are already getting burned financially by agent costs, and are pulling back hard

Real dollar figures and rollback data show this isn't theoretical — it's already causing budget blowouts and program cancellations.

"Uber reportedly exhausted its entire 2026 AI budget in four months. Microsoft is reported to have ended its Claude Code licences after a pilot that began in December 2025." "the rollback rate reached 81% at companies with mature governance frameworks, which is to say the organisations paying closest attention pulled back hardest."

Most token waste is a design problem, not an efficiency problem

The article's central thesis: uncontrolled costs stem from undefined workflows (no stop conditions, no scoped context) rather than inherent AI inefficiency.

"Most runaway token bills aren't an efficiency issue at all, they're a design issue... The system doesn't know what's important, so it sends everything." "An agent's invoice was never a record of how hard it worked. It is a record of everything nobody bothered to decide."

There's a clear hierarchy of fixes: free/no-tradeoff changes first, judgment-based changes second

The piece structures cost-cutting into two tiers — lossless technical fixes (caching, formatting, batching) versus fixes requiring evals (model routing, compression, thinking budgets).

"Some changes leave the model receiving precisely what it received before. Those go first, because there is nothing to weigh up." "From here on, the changes need some judgment, plus a way to test them."

2. Contrarian Perspectives

Shortening prompts/output is not a reliable cost-saving lever — it can backfire entirely. This directly contradicts the common instinct (used across the industry) to simply cut context to save money.

"Enterprises do the same thing at scale. When the API bill gets scary, they shorten prompts, cap output, and switch to cheaper models... Less context was never the fix."

Measuring or rewarding AI usage volume (tokens/activity) is counterproductive. Internal leaderboards and adoption metrics that reward heavy usage actively create the waste they're meant to track.

"Leaderboards that celebrate whoever burned the most tokens reward activity over good judgment, and you end up with exactly what you measured."

Automated/algorithmic compression can silently break intended behavior without any warning from offline metrics. An AI-driven prompt-compression pass halved a prompt size and passed evals, but broke production parallelism — showing eval-passing isn't sufficient validation.

"The rewrite had taken a gentle suggestion about running tasks in parallel and turned it into a strict rule... An untested behaviour can be deleted by a compression pass without anyone noticing."

3. Companies Identified

GitHub — Software collaboration platform / Copilot coding agent maker Why mentioned: Ran and published the central case study on agent cost engineering, testing token-shortening techniques and reporting real production results.

"GitHub publishes its agent cost engineering, which makes it one of the few places where any of this gets tested rather than asserted."

OpenClaw — AI coding agent system built by Peter Steinberger Why mentioned: Extreme example of agentic token consumption at scale, cited by academic researchers as a case study.

"His team spent $1.3 million on tokens in one month, 603 billion of them, across 100 coding agents run by three people."

Uber — Ride-hailing/logistics company Why mentioned: Cited as an enterprise example of agent costs outpacing budget planning.

"Uber reportedly exhausted its entire 2026 AI budget in four months."

Microsoft — Technology company Why mentioned: Cited as an example of enterprise rollback of agentic AI tooling after cost concerns.

"Microsoft is reported to have ended its Claude Code licences after a pilot that began in December 2025."

Upwork — Freelance marketplace (sponsor) Why mentioned: Case study/product placement for its new MCP connector integrating freelancer sourcing directly into Claude/ChatGPT workflows.

"Upwork's new MCP connector for Claude and ChatGPT collapses that into one conversation... One client had Claude read 50 profiles against his job post. It took minutes. He'd budgeted a week."

Codex — AI coding agent (OpenAI) Why mentioned: Named alongside OpenClaw as a system studied specifically for high token consumption.

"There is a recent position paper on token reduction from Harvard, MIT and Northeastern singles out OpenClaw and Codex as the systems worth studying, simply because of how many tokens they burn through."

4. People Identified

Peter Steinberger — Creator of OpenClaw Why mentioned: Provided a concrete, extreme data point on real-world agentic token spend, illustrating the scale of the cost problem.

"His team spent $1.3 million on tokens in one month, 603 billion of them, across 100 coding agents run by three people."

Ruben Dominguez — Author/writer of the newsletter piece Why mentioned: Byline author; frames the entire analysis and GitHub case study.

5. Operating Insights

Order your prompts for caching, and audit for cache-breaking leaks. Put static content (system prompt, schemas, reference docs) first and dynamic content (queries, timestamps, session IDs) last — a stray dynamic detail in the static block causes full reprocessing at full price.

"The catch is that a single dynamic detail, like a timestamp, a session ID or a user name, can slip into the static block without anyone noticing. When it does, everything after it gets reprocessed at full price."
Route by task difficulty, not by default to the strongest model. A simple classifier sending routine work to cheaper models can cut bills dramatically with minimal quality risk.

"A simple classifier pushes 60% to 70% of production traffic into the bottom tier, where tokens run 10 to 50 times cheaper... One team shipped a router and watched the monthly bill fall 58% that same week."
Only compress predictable, low-risk content (logs, build output) — leave source code/diffs/raw script output untouched, and use "how often the agent reopens the original" as your real signal of whether compression is safe, not offline benchmarks.

"Git diff was in the compression set at first. It was taken out after benchmark tasks showed agents reopening the originals... The agent can always go back to the full original, and how often it does is a better sign of whether compression works than any offline metric."

6. Overlooked Insights

PDF ingestion is a massive, invisible token tax. Converting documents to plain text before feeding them to a model can cut costs by an order of magnitude with no visible tradeoff — a tactic easy to miss amid discussion of prompts and history.

"One 42 page report cost 84,000 tokens per call as a PDF and 9,500 after conversion."
Background/async task-completion handling quietly wastes model calls. Batching completion notices for parallel tasks is a small, easy fix that's rarely discussed compared to headline levers like caching or model routing.

"Two tasks running at the same time meant four model calls before work could resume. Batching those completions cut it down to one, which saved about 2.3% without compressing, summarising or leaving out anything."