Why AI Harnesses Beat Models: Cost & Performance Impact

Key Insights

• Harness matters more than model: GPT-5.5 scored 61.5% in Codex but 87.2% in Cursor—a 25.7-point jump from the runtime alone
• Input tokens dominate costs: Input makes up 86-98% of LLM traffic on OpenRouter, so controlling input costs is the real battleground
• Smart caching saves 40-80%: Intelligent prompt caching across long-horizon agent sessions reduced costs by 41-80% and improved time-to-first-token by 13-31%
• Third-party harnesses can match first-party optimization: Cursor's harness matches Claude Code's caching technique—and achieves better scores for both models

The Harness Is More Powerful Than You Think

Endor Labs' Agent Security League study revealed a surprising truth: the infrastructure running an AI model matters more than the model itself. When researchers tested GPT-5.5 in OpenAI's native Codex harness, it achieved 61.5% functional correctness. The same model in Cursor's harness? 87.2%—a 25.7-point improvement.

Anthropic's Claude Opus 4.7 followed the same pattern: 87.2% in Claude Code, 91.1% in Cursor. Both frontier models performed better in a competitor's harness than in their maker's own implementation. This wasn't a flaw in the native systems; it was evidence that harnesses drive performance as much as raw model capability.

Where Costs Really Hide: Input Token Control

Most companies focus on model choice, but they're missing the actual cost driver. On OpenRouter, input tokens account for 86-98% of total token volume. While output tokens cost 5x more per token, sheer input volume makes input the dominant expense at scale.

This shifts the optimization problem entirely. The model doesn't control input costs—the harness does. Harnesses decide which context to send, which information to retrieve, and how to structure prompts. When context repeats across queries—which it does in most agent workflows—intelligent caching becomes a massive lever.

A study across 500 long-horizon agent sessions found that strategic caching reduced costs by 41-80% while cutting time-to-first-token by 13-31%. The winning strategy? Cache only the stable prefix (system prompts, shared context) and place dynamic content after the cache breakpoint. Savings scaled linearly from 500-token to 50,000-token prompts.

Cursor vs. Claude Code: How Third-Party Systems Win

Claude Code achieves impressive cache hit rates by design: 96% in real sessions, shared system-prompt caches across users, and forked sub-agents at 99% byte-identity for 90% cost savings. This looks like a first-party advantage.

But Cursor's harness replicates Claude Code's technique stack: dynamic tool fetching, priority-based prefix assembly, and two-tier caching. The result? Both models scored higher in Cursor than in their native environments. This proves that cache discipline lives in the harness architecture, not exclusively in first-party integration.

Co-designing the harness, cache API, and model together helps—but a well-engineered third-party harness can match or exceed those results.

Conclusion

The era of model-centric AI is ending. The harness—once dismissed as a simple wrapper—is now the jockey pushing AI performance further than the model breeder imagined. If you're optimizing for cost and speed, focus on input control through intelligent caching, concise information retrieval, and strategic context placement. The model choice matters less than you think.

---

Original source: Aftermarket Harnesses

powered by osmu.app