Cut AI coding token spending by 90% using smart model routing. Learn how to delegate I/O work to cheaper models while keeping expensive reasoning for complex...
Cut AI Coding Token Costs by 90%: The Smart Routing Strategy
Overview
AI coding agents spend most of their time on routine I/O work—reading files, generating boilerplate, updating documentation—not actual thinking. This consumes expensive frontier model tokens on tasks that cheaper alternatives handle just as well. By 2028, AI coding costs are expected to surpass the average developer's salary, with many engineering teams already spending $200–$2,000 per developer monthly on tokens. A strategic solution exists: model routing through declarative agents that handle grunt work cheaply while reserving expensive models for problems requiring genuine reasoning.
Key Insights
- 90% token savings possible: Routing routine I/O work away from expensive models reduces token consumption dramatically
- Cost crisis emerging: AI coding token spending per developer is projected to exceed annual salaries by 2028
- Two-mode approach works: A simple pattern of delegating file reading and code generation solves most token waste
- No infrastructure required: Ephemeral agent modes eliminate the need for platform teams, new subscriptions, or server management
- Gradual degradation: System works even if delegation fails; hooks still block expensive reads
The Problem: Token Waste on Routine Work
Most AI coding work isn't reasoning—it's I/O. Reading multiple files to answer a single question. Generating test files that match twenty existing patterns. Updating documentation. These tasks consume thousands of tokens and almost zero reasoning capacity, yet they're fed to frontier models vastly overqualified for the job.
A quarter of engineering leaders burn $200–$500 per developer per month on tokens. Some spend well over $2,000. The tooling pays for itself only if you stop wasting expensive tokens on work that doesn't require them. The math is clear: by 2028, AI coding costs will blow past the average developer's salary, making inefficient token usage unsustainable.
Solution: Two Modes for Delegation
The fix doesn't require new infrastructure, platform teams, or additional subscriptions—just two declarative agent modes. AiKA Modes in Portal by Spotify provide ephemeral runtimes (like AWS Lambda for agents) where you define instructions, pick a model, set parameters, and attach tools. Portal handles deployment; you handle no infrastructure, API keys, or long-running servers.
Mode 1: Bulk Reader
Purpose: When you'd otherwise spend expensive tokens reading multiple large files for one answer.
How it works: Reads provided files and answers the question concisely using structured bullets only—no prose or preambles. Every bullet leads with the exact name, type, or line number; nested bullets provide details. The model skips anything not directly asked.
Token benefit: Instead of an expensive model reading five files to answer one question, a cheaper model handles the I/O and returns just the structured facts needed.
Mode 2: Code Writer
Purpose: Tests, config scaffolding, type stubs, or any output where the pattern is predictable from existing code.
How it works: Takes a spec and a reference file, generates code matching existing patterns exactly, and outputs only the code—no markdown fences or explanations unless requested. The reference file is mandatory; without it, the worker generates disconnected code that fits nothing in your project.
Token benefit: Code generation never enters the expensive model's context. The expensive model requests the generation; the worker delivers it; the expensive model never sees it.
Enforcement: Three-Layer Routing
Simply telling an AI agent to use cheaper models doesn't work—it ignores advisory rules. The solution uses three enforcement layers:
Layer 1: Hooks
PreToolUse hooks fire before every tool call, blocking expensive work before it starts:
- check-file-size blocks reads on files exceeding a configurable threshold (default: 350 lines), redirecting to bulk-reader. Targeted reads pass through because the model already knows which section it needs.
- check-bash-read catches piped commands on large files (cat, head, tail, less, more), distinguishing targeted reads from bulk reads.
The threshold is adjustable via SHUNT_MIN_LINES environment variable in your shell profile or .claude/settings.json.
Layer 2: Scripts
Bash wrapper scripts invoke Portal CLI actions with named arguments, handling request building, error unwrapping, and token usage reporting. Modes are resolved by Portal in priority order: your custom version first, then team versions, then public ones.
- bulk-read: Wraps files in XML tags and sends them to bulk-reader mode. Subsequent calls reuse the same files for free because the corpus never enters the expensive model's context.
- code-write: Sends spec and reference file to code-writer, strips markdown from output, and optionally writes directly to disk. The expensive model never sees the generated code.
Layer 3: Skills
Markdown skill files tell the model when and how to call the scripts. When hooks block reads, redirect messages point to the /bulk-reader skill showing exact syntax. This layering degrades gracefully—even if the model doesn't read the skill, hooks still block expensive reads.
The Benchmarks
Testing on a Java monorepo across four scenarios shows mean bulk-read savings around 90%—measuring tokens the expensive model would consume reading files directly versus consuming the cheaper model's structured summary.
Code-write is harder to measure purely in tokens because without routing, the expensive model both reads reference files and generates output as expensive output tokens. With routing, code goes straight to disk; the expensive model never sees it.
What Routing Can't Do
Editing: Worker model summaries lack reliable line numbers. If reasoning based on analysis requires edits, the expensive model must still read the specific section directly.
Reasoning: The worker model surfaces patterns but misses subtle issues (like thread-safety bugs). Reasoning, debugging, architectural decisions, and safety-critical code stay with the expensive model.
Small reads: Latency—10–30 seconds per round-trip, capped at 30 seconds per invocation—makes delegation counterproductive for small files. The line threshold exists precisely to avoid this overhead.
Why Modes Are the Real Power
The plugin is a practical tool, but AiKA modes represent the deeper value: model routing becomes a configuration problem, not a systems problem.
- Reusable: The same modes work across every project and tool that can shell out to Portal CLI
- Shareable: Public modes require zero setup; custom versions automatically take precedence
- Composable: Add modes for documentation, code review, translations—each one is a configuration change
- Decoupled: Swap models, adjust prompts, add tools—the routing plugin doesn't change
Getting Started
- Install both plugins from the
spotify/portal-ai-pluginsmarketplace - Run
/portal:setupin a Claude Code session to authenticate - Bulk-reader and code-writer modes are already public—start delegating immediately
The system works across projects with zero customization. Fork public modes to customize, and your version takes precedence automatically.
Conclusion
AI coding token costs are unsustainable without smarter routing. By separating I/O work from reasoning work and delegating the former to cheaper models, you cut token waste by 90% without building infrastructure. Start with the public bulk-reader and code-writer modes today—the routing enforces itself, and your token bill shrinks immediately.
원문출처: Portal by Spotify cut my Claude Code token usage by 90% | Spotify Engineering
powered by osmu.app