Memory Architecture for AI Agents: Building Smarter Systems

Key Insights

• Preflight retrieval loads only relevant skills from a library into context, replacing raw token limits with intelligent memory selection
• Local execution handles 80% of routine tasks on Ornith 35B, routing complex queries to frontier models for efficiency
• Watchdog monitoring logs all decisions and skill usage, enabling overnight improvements and self-improvement loops
• Self-improving systems can reach plateaus where only genuinely new exceptions require human intervention

The Problem: Context Size Alone Isn't Enough

Traditional AI agents face a fundamental constraint: context windows fill quickly with raw data. When an agent receives a request like "Summarize the Q3 board deck," 200,000 raw tokens of emails, PDFs, and chats become a bottleneck.

The solution isn't bigger context windows. It's smarter memory architecture.

How Preflight Retrieval Works

Preflight functions like a pilot's pre-takeoff checklist. Before executing any task, the agent inspects its skills library—a collection of ~90 indexed workflow files—and selects only the skills relevant to the current request.

Instead of loading all available data, the preflight step retrieves the right subset. These consolidated memory "skills" are then loaded into the context window for the local model to execute. This filtering mechanism transforms memory from a fixed size constraint into an intelligent selection process.

Local Execution with Strategic Routing

The Ornith 35B model—a locally-hosted, open-weight model running on Apple Silicon via Ollama—executes on the loaded context. The system keeps routine tasks local about 80% of the time.

Tasks that exceed the local model's capability route to frontier models (larger, more powerful systems). This hybrid approach balances cost, latency, and quality.

The Watchdog: Building a Self-Improving Loop

A watchdog process monitors which skills are loaded, which decisions are made, and overall success rates. Every preflight decision and skill invocation is logged as a named, versioned artifact.

Overnight, asynchronous inference processes the day's trail of decisions. The system identifies:

• Which new skills should be developed
• Which parts of existing skills should become deterministic code (like calendar scheduling with Rust instead of an LLM)

The skills library is rewritten and the system restarts in a self-improving loop. On one recent day, the watchdog suggested no improvements—hinting that the system had reached a plateau where only genuinely new exceptions need human intervention.

Conclusion

Memory architecture—not context size—determines how well AI agents scale. By combining preflight retrieval, local execution, and overnight watchdog monitoring, agents can become genuinely self-improving systems that adapt without constant human tuning. The result: smarter agents that learn from their own decisions.

Ready to explore memory-first AI design? Start by mapping your skills library and logging decisions systematically.

---

Original source: The AI Preflight Check

powered by osmu.app