AI Harnesses Beat Models: Why Framework Matters More Than Algorithm

Key Insights

• Harnesses drive larger performance gains than model upgrades. GPT-5.5 scored 61.5% in Codex but 87.2% in Cursor—a 25.7-point improvement from framework alone.
• Input tokens dominate costs. 86-98% of LLM traffic is input tokens, making cache discipline the primary cost lever—not model selection.
• Strategic caching cuts costs by 40-80%. Studies show that intelligent prompt caching across long-horizon agent sessions reduces spending 41-80% and speeds time-to-first-token by 13-31%.
• Third-party harnesses can match first-party performance. Cursor's cache techniques match Claude Code's, proving framework excellence isn't vendor-locked.

---

The Harness vs. Model Performance Gap

Endor Labs' Agent Security League tested frontier models across different harnesses in the same week. The results shattered conventional assumptions: the framework matters more than the model.

GPT-5.5 scored 61.5% functional correctness in OpenAI's native Codex harness—but jumped to 87.2% in Cursor's harness. Anthropic's Opus 4.7 similarly scored 87.2% in Claude Code but 91.1% in Cursor.

Both frontier models performed better in a competitor's harness than in their maker's native environment. This 25+ point swing from runtime alone reveals a fundamental truth: the harness is the jockey; the model is the horse.

Why Input Tokens Dominate Your LLM Bill

Most practitioners assume output tokens drive costs. Reality is starkly different.

Input tokens represent 86-98% of token volume on OpenRouter, while outputs make up the remainder. Although output tokens cost 5x per token, input volume is so massive that it dominates total spend at scale. Controlling input costs, therefore, is where real savings live—and only the harness controls that.

Models don't decide which context gets sent. Harnesses do. When the same context repeats across queries—a common pattern in agent systems—intelligent caching extracts dramatic savings:

• 41-80% cost reduction across 500+ long-horizon agent sessions
• 13-31% faster time-to-first-token performance
• Linear scaling from 500-token to 50,000-token prompts

The winning strategy: cache only the stable prefix and place dynamic content after the cache breakpoint, avoiding unnecessary cache invalidation.

How Top Harnesses Achieve Superior Performance

Cursor's harness implements three cache techniques that also power Claude Code:

1. Dynamic tool fetching – Load only relevant context when needed
2. Priority-based prefix assembly – Arrange context by likelihood of reuse
3. Two-tier caching – Separate stable and dynamic layers

This is why Opus 4.7 outperformed Claude Code's native environment in benchmarks—Cursor's framework implementation was simply more disciplined.

Beyond caching, harnesses optimize information retrieval: deciding which code files to load, which style guides to include, which sections of briefing documents to analyze. More concise and precise context lowers costs and improves quality.

The First-Party vs. Third-Party Trade-Off

First-party co-design (model + harness from the same vendor) offers real advantages. Claude Code, for example, achieves 96% cache hit rates in real sessions by sharing system-prompt caches across users and building forked sub-agents at 99% byte-identity for 90% savings.

But co-design isn't a moat. The discipline that drives those results lives in the harness architecture, not the model. Third-party harnesses like Cursor prove that excellent cache strategy, intelligent prefix assembly, and two-tier caching can match or exceed first-party performance without proprietary system access.

---

Conclusion

The AI stack's new power dynamic isn't model vs. model—it's harness vs. harness. As frontier models converge in capability, framework choices determine real-world performance and cost. Teams optimizing for production efficiency should prioritize harness architecture, cache strategy, and prompt caching discipline over chasing marginal model improvements.

---

Original source: Aftermarket Harnesses

powered by osmu.app