Codex Harness Engineering: From Design to Production Deployment

핵심 요약

Harness engineering creates repeatable AI workflows by structuring agent behavior through rules, memory, and automated processes
Six core Codex components (agent.md, Memory, Skills, MCP, Hooks, Sub-agents) work together to enforce standards, define procedures, and control execution
Practical workflow: Interview → Planning → Project Setup → Development → Code Review → Deployment ensures consistency and reduces mid-project changes
GitHub integration and ** Vercel deployment** enable seamless CI/CD with automated code reviews via Codex agents
Production-ready service (Commit Hero) demonstrates real-world implementation of harness principles with API integration and AI features
Context optimization through nested agent.md files, skill-based progressive loading, and documentation planning maximizes token efficiency
Code review automation at multiple stages (local → pull request → production) eliminates bottlenecks and ensures security

Understanding Harness Engineering and Its Critical Role

Harness engineering represents a fundamental shift in how we work with AI agents. At its core, a harness is a structured framework that fixes the way an AI agent operates, ensuring it works consistently according to predefined rules and workflows. Without a harness, relying solely on prompts creates massive inefficiencies—when building applications or complex services, raw prompts alone provide insufficient guidance for reliable agent behavior.

The importance of harness engineering has grown exponentially because it solves three critical problems. First, it creates repeatability: once you define a harness, the agent performs tasks the same way every time, eliminating unpredictable variations. Second, it enables ** separation of compulsion and autonomy**: you can force certain behaviors through enforcement mechanisms while allowing the agent flexibility in areas where innovation matters. Third, harnesses become ** accumulated knowledge at team level**: instead of each team member writing custom prompts, everyone benefits from shared, refined harness components that embed best practices and domain expertise.

The stakes are particularly high in production environments. An AI agent without proper harness structures can make costly mistakes, violate security protocols, or produce inconsistent outputs that damage user trust. This is why organizations investing in harness engineering see dramatically improved deployment safety and developer productivity. The investment upfront in designing a robust harness pays dividends across every project using that same harness.

The Six Core Harness Components in Codex

Codex provides six primary harness components, each serving distinct purposes in creating a complete agent control system:

1. Agent.md: The Foundation Rule System

Agent.md functions as Codex's rule file—a definitive reference document that agents read and apply continuously throughout their operations. Think of it as constitutional guidelines that govern all agent behavior within a project or globally. The power of agent.md lies in its hierarchical scope system: you can create global agent.md files in the ~/.codex/ directory that apply across all projects, or project-specific agent.md files in your repository root that apply only within that project context.

The hierarchical design of agent.md is remarkably efficient. Codex reads the global default directives first, then sequentially combines directives from the project root down to the current directory. This means you can create nested agent.md files at different folder levels—placing one in your root directory for general rules, then creating specialized ones in subdirectories for domain-specific workflows. This nested structure is crucial for context optimization: instead of loading all rules at once (which wastes tokens), Codex only reads the agent.md relevant to the specific folder you're working in. For teams, agent.md files should always be version-controlled and shared, becoming the single source of truth for how agents should behave.

A well-crafted agent.md prevents countless errors by being explicit about requirements, constraints, and procedures before work begins. Many developers underestimate this file's importance, but experienced harness engineers emphasize that defining rules upfront is far more efficient than fixing agent mistakes afterward.

2. Memory: Learning from Experience

Memory is disabled by default but represents one of the most valuable harness components when activated. Unlike traditional session-based AI interactions where each conversation starts fresh, enabled Memory allows Codex to extract and retain useful contexts from previous conversations and tasks, storing them for retrieval in future sessions.

When Memory is active, Codex automatically analyzes your conversation context while you work and extracts key information: your preferences, repetitive workflows, your tech stack choices, project management patterns, and common mistakes. These memories accumulate in a dedicated memories folder within Codex's configuration directory. When you enable the "use memory" option in settings, the agent can automatically find and retrieve appropriate memories from past sessions, providing contextual continuity across multiple projects and time periods.

This is particularly powerful for individuals and teams with consistent development patterns. For example, if you repeatedly prefer certain architecture patterns, naming conventions, or error-handling approaches, Memory learns these preferences and applies them automatically. The system essentially builds a personalized knowledge base of your professional practices, reducing the need to repeatedly explain your standards. However, since this remains an unofficial feature, you must explicitly activate it in your configuration file.

3. Skills: Modular, Reusable Workflows

Skills package repetitive procedures and domain expertise into reusable units that avoid redundancy while maximizing efficiency. While they might seem similar to prompt templates, skills are substantially more powerful: they combine prompts, scripts, data files, and other assets into a cohesive module designed for specific tasks.

The structural elegance of skills comes from their progressive disclosure pattern. When Codex loads a skill, it doesn't immediately consume massive context by loading all contents. Instead, it only initially retrieves the "front matter" section from skill.md—which contains essential metadata like skill name and description. This requires minimal tokens. As the skill is used, Codex progressively loads additional components: reading the full skill.md, then references, then scripts, and finally sub-agents, only as needed. This progressive loading approach is radically more efficient than traditional monolithic rule files.

For developers with repetitive workflows, skills are essential. Instead of writing similar prompts repeatedly, you create a skill once and reuse it across projects. Skills also enable specialization: you can create skills for specific domains (API documentation review, security auditing, database schema design) that bundle domain-specific expertise, making that expertise available across your entire team instantly. The decision to create a skill is straightforward: whenever you find yourself writing similar prompts multiple times, convert that workflow into a skill.

4. MCP (Model Context Protocol): External System Integration

MCP is a standardized protocol—originating from Anthropic—that enables LLM models to connect to external systems and tools that agents cannot natively handle. MCP acts as the bridge between your AI agent and external APIs, services, and platforms. Common examples include Slack integration (allowing agents to read messages, analyze content, and respond), email systems (agents can read, compose, and send emails), GitLab or GitHub (agents can manage repositories, create issues, and review code), and custom business systems.

In Codex, MCP integration expands the agent's capabilities beyond built-in functions. Rather than agents being limited to local file operations and direct API calls, MCP allows them to interact with enterprise systems seamlessly. For example, an agent could monitor Slack for specific messages, automatically analyze them, and escalate critical issues to the appropriate team. Or in development workflows, agents can use MCP to automatically create GitHub issues, manage pull requests, and coordinate with deployment systems.

The advantage of MCP is standardization: since it's a protocol-based approach, many popular services have standard MCP implementations, making integration straightforward without custom coding. However, integration still requires some initial setup to authenticate connections and define agent permissions.

5. Hooks: Enforcement Through Programming Logic

Hooks are runtime automation and guardrails that programmatically enforce specific behaviors at defined event points in the agent execution loop. Unlike agent.md rules that rely on the agent reading and making judgments (which isn't always reliable), hooks use actual programming logic to forcefully trigger actions.

Hooks activate at various lifecycle points: when a session starts, when a user submits a prompt, before the agent uses a tool, or when a turn is completely finished. At each point, you can inject specific commands or validations. For example, you might use a hook to prevent the agent from accessing sensitive environment variables (a security guardrail), or to automatically save conversation state at session end, or to validate that generated code passes linting before execution.

The critical distinction is that hooks operate through programming logic, not suggestions. This means enforcement is absolute: if a hook blocks an action, the agent cannot bypass it through reasoning. This is essential for security-sensitive operations, compliance requirements, or preventing costly mistakes. Hooks are particularly valuable for teams where you need to guarantee certain behaviors regardless of the agent's reasoning process.

6. Sub-agents: Parallel Task Delegation

Sub-agents enable you to create separate specialized agents with independent contexts that work in parallel on delegated tasks. While your main agent handles overall coordination, sub-agents can work simultaneously on distinct specialized tasks, dramatically accelerating complex projects.

Sub-agents are powerful for several reasons. First, they parallelize work: instead of one agent sequentially handling ten tasks, you can create ten sub-agents working simultaneously. Second, they maintain ** context independence**: each sub-agent has its own isolated context window, meaning specialized work doesn't pollute the main agent's context. This isolation is valuable for code review verification: you create a sub-agent whose sole task is reviewing code generated by the main agent, providing unbiased feedback from a completely separate perspective. Third, sub-agents enable ** specialized focus**: you can create a sub-agent with specific expertise (security auditing, performance optimization, documentation) that the main agent can call when needed.

The parallel execution capability makes sub-agents essential for large projects. Rather than waiting sequentially for one agent to complete multiple tasks, you distribute work across specialized sub-agents and merge results afterward.

Codex Configuration Paths: Where Everything Lives

Codex maintains configuration and project data in two primary locations, and understanding this distinction is essential for advanced harness engineering:

Global Configuration Path (~/.codex/): This directory in your home folder stores Codex's core settings, authentication credentials, execution history, logs, memory files, hooks, and other global execution state. This is where global agent.md files belong, where memory accumulates across projects, and where system-level configurations are managed. The config.toml or config.yaml file here contains your settings for Memory activation, intelligence levels, speed preferences, and connected services.

Project Configuration Path (~/.agent/): This directory stores Codex's project-level skills and global skill libraries. Skills installed here become available across all projects. This is where you install shared domain expertise and commonly-used workflow skills, making them instantly accessible without installation in each project.

Understanding these paths is crucial because it informs where you should store different harness components. Global rules that should apply everywhere go in ~/.codex/agent.md. Project-specific rules go in your repository's ./agent.md. Shared skills go in ~/.agent/, while project-specific skills belong in the project directory. This separation prevents configuration bloat and ensures that each component lives where it's most useful.

Designing and Implementing an Effective Harness Strategy

Rather than attempting to design a perfect harness from the beginning, experienced developers recommend building harnesses incrementally. This iterative approach acknowledges that you often don't fully understand requirements until you start working, and that trying to perfect everything upfront wastes effort.

The Recommended Harness Implementation Order:

First, define agent rules by creating a clear agent.md that specifies what the agent must always follow. These are your non-negotiable standards. Second, ** implement skills** when you identify prompts you're repeatedly writing or workflows worth formalizing—convert these into skills to avoid redundancy. Third, ** add external tool connections** using MCP when you need to integrate systems beyond the agent's native capabilities (Slack, email, GitHub, custom APIs). Fourth, ** implement enforcement mechanisms** using hooks and sub-agents for verification, approval workflows, and tasks requiring complete isolation or specialized review.

This progression ensures you're building complexity only when needed. Many developers make the mistake of trying to create perfect hooks and sub-agents immediately, when simple rules in agent.md would suffice initially.

The Overarching Goals of Harness Engineering can be summarized concisely:

Set standards through agent.md and Memory (defining what should always be true)
Define procedures through skills and MCP (automating workflows and connecting systems)
Control execution through hooks and sub-agents (enforcing requirements programmatically)

Practical Workflow: Building Commit Hero from Interview to Production

The real power of harness engineering becomes apparent through practical implementation. The "Commit Hero" service demonstrates this complete workflow: a web application that accepts a GitHub username, analyzes the developer's commit history, and generates RPG-style character cards displaying the developer as a game character with class, level, stats, and abilities.

Phase 1: The Interview - Clarifying Ambiguous Requirements

Most developers underestimate how difficult it is to specify requirements clearly. You often don't know exactly what you want until you start exploring. This is why the interview phase is invaluable: instead of you asking questions, the AI asks you the necessary questions, forcing you to think deeply about your actual needs.

Using the "Deep Interview" skill (available in the lecture materials), the process begins with a rough prompt: "I want to create a viral service that developers can use. When a user accesses and enters a GitHub username, it analyzes their commit history and displays it as an RPG game card."

The AI then asks clarifying questions:

"What's your primary success criterion? Fun shareable cards? Accurate developer analysis? Or virality itself?" → Decision: Fun and shareable, designed for viral sharing.
"What tone should cards use? Serious RPG fantasy? Developer meme humor? TCG collectible style?" → Decision: Developer meme humor.
"MVP data input method: username-only or OAuth login?" → Decision: Username-only for instant accessibility.
"Card elements: character stats (class, level, skills) or analysis report format?" → Decision: Both—combine character card with analysis report.
"Sharing method: URL-based or image download?" → Decision: Support both options.
"Card generation: rule-based (fast, deterministic) or AI-generated (creative, slower)?" → Decision: Rule-based for MVP, designed for future AI expansion.
"Deployment architecture: static client, Vercel functions, or serverless?" → Decision: Next.js with Vercel, using Bun package manager.

This interview transforms vague intent into concrete specifications. What began as "create a viral service" becomes a detailed specification: a Next.js application with meme humor, rule-based GitHub analysis, downloadable cards and shareable URLs, deployed to Vercel. The interview eliminates ambiguity that would otherwise cause mid-project changes.

Phase 2: Planning Documentation - Creating Reference Architecture

Once interview decisions are finalized, Codex generates a comprehensive plan document in Markdown format. This document serves as a shared reference that keeps the agent consistent throughout development, independent of context window degradation. The plan includes:

Clear success criteria and scope definition
Excluded features (explicitly stating what's NOT in scope)
User workflow diagrams (username input → analysis → card display → sharing)
API contract specifications (GitHub API calls, response structures, error handling)
Frontend component requirements (form page, card display page, download functionality)
Data analysis rules (how to map GitHub metrics to RPG stats)

The documentation step is underrated but critical. As development progresses, context windows fill up, and without external documentation, agents can lose track of original intent. The plan document remains accessible as a reference file, keeping work aligned with initial design.

For complex plans, converting from Markdown to HTML dramatically improves human review. While both formats are machine-readable, HTML visualization makes it instantly obvious whether the plan matches your vision. A 5-minute HTML review can catch architectural misalignments that would otherwise require days of recoding.

Phase 3: Project Structure - Establishing Development Guidelines

Before coding begins, you establish the foundational structure and define agent.md rules. Using a master prompt (provided in lecture materials), you create an agent.md that specifies:

Directory structure conventions
Naming standards for files, functions, and variables
Technology choices and their rationale (Next.js over alternatives, Bun over npm)
Code style expectations (formatting, comments, error handling)
Security requirements (no hardcoded credentials, environment variables only)
Testing expectations (unit tests, integration tests, coverage thresholds)

This agent.md acts as constitutional law for the project. Every file created and every decision made should align with these rules. When the agent has clear rules, it generates consistent code automatically.

Phase 4: Implementation - Building the MVP

Starting with mock UI implementation (before backend integration) allows you to validate the user interface and flow without external dependencies. Codex generates the Next.js component structure, creates the main landing page with a GitHub username input form, and builds the card display page showing mock character data.

The Codex in-browser feature (Ctrl+Shift+B) lets you instantly see rendered components within the IDE, and Codex agents can interact with this browser to verify their own work. This creates a tight feedback loop: agent generates code → agent opens browser → agent verifies rendering → agent fixes issues.

Then API integration follows: connecting to GitHub's public API to fetch real user data. The implementation includes:

REST API routes that call GitHub API (without requiring user authentication)
Data analysis rules that score developer activity (commits, languages, repositories)
Card generation logic that transforms scores into RPG stats and humorous class assignments

Phase 5: Code Review - Multi-Stage Verification

Rather than pushing code directly to production, professional development uses code review at multiple stages:

Local Code Review: Before committing, request code review within Codex itself. You can either open a new chat session for unbiased review perspective, or use sub-agents for parallel review from completely independent contexts. This catches security issues, performance problems, and architectural flaws locally.

Pull Request Code Review: Once local changes pass review, create a GitHub pull request. Connect Codex's code review bot (available through Codex Cloud) to your repository. The bot automatically reviews every pull request, checking for security vulnerabilities, code quality, and adherence to standards.

Human Review: After AI review clears issues, human developers review for architecture validation and security considerations. Humans aren't reading line-by-line code (which creates bottlenecks); they're reviewing overall design and validating that AI suggestions are sound. This multi-stage approach eliminates the human-as-bottleneck problem.

Phase 6: Deployment - Seamless CI/CD

Once code review passes, push to the main branch. Vercel automatically detects the push, triggers deployment, and makes the service live. This automation means deployments are fast, consistent, and no longer manual bottlenecks.

Importantly, for this MVP deployment, only rule-based analysis runs (no AI features yet). This keeps the service fast and free of API costs during testing.

Phase 7: Enhanced Features - Parallel Development with Sub-Sessions

After the MVP deploys successfully, you're ready to add AI features. Rather than continuing in the same session (which would pollute context and lose focus), you create a new worktree—Git's feature for parallel branch development. In this new session, you:

Add OpenAI GPT API integration
Implement structured output formatting
Create more creative, dynamic character descriptions based on AI analysis
Test thoroughly before merging back to main

Using worktrees means your new features develop in completely isolated folders, preventing conflicts with main branch work. Once thoroughly tested, you merge these enhancements back to main, creating a new deployment.

Leveraging Codex Settings for Optimal Performance

Codex provides configuration options that dramatically affect both performance and cost:

Intelligence Level: Set to "Very High" for complex, long-context tasks where you want the agent to think deeply before acting. Set to "Medium" for straightforward tasks where spending more on thinking doesn't add value. Higher intelligence consumes more tokens but produces more stable, thoughtful results.

Speed Mode: Normal speed is suitable for background work and parallel sessions (you're doing other things anyway). Speed mode (1.5x or 2x) accelerates responses but increases token consumption proportionally. Use it for interactive sessions where you're watching and waiting. For large projects, normal speed often completes faster overall because the agent thinks more carefully and avoids mistakes requiring rework.

Model Selection: Codex currently uses GPT-4o, which represents state-of-the-art performance. Choosing the latest available model ensures best-in-class capabilities for harness design and complex coding tasks.

These settings should be adjusted per-session based on task type. A deep architecture planning session warrants "Very High" intelligence + normal speed. Debugging a simple issue might use "Medium" + speed mode.

Advanced Patterns: Nested Agent.md and Context Optimization

For large projects spanning multiple domains, creating nested agent.md files dramatically improves context efficiency. Place a general agent.md at your repository root with project-wide standards. Then create domain-specific agent.md files in subdirectories: one in /backend for server-side standards, one in /frontend for client-side conventions, one in /database for schema design rules.

When Codex works in the /backend directory, it reads both the root agent.md AND the /backend/agent.md, combining all relevant rules. Critically, it doesn't load the /frontend/agent.md, preserving context for actual work. This hierarchical approach maximizes rule coverage while minimizing context waste.

Memory serves a similar optimization purpose. Rather than repeating your preferences, tech stack choices, and common decisions in every agent.md, Memory learns these patterns and applies them automatically across sessions.

Multi-Agent Code Review: Adversarial Perspectives

Professional development increasingly uses multiple independent agents for code review. The logic is sound: an agent reviewing its own code tends toward confirmation bias, accepting its own reasoning. However, an agent reviewing code in a completely separate session, with no context of the development discussion, catches issues the original agent missed.

For example, after your main agent completes a feature, you could:

Create Sub-agent A (security specialist) to review for vulnerabilities
Create Sub-agent B (performance specialist) to review for optimization
Create Sub-agent C (code quality specialist) to review for maintainability

Each operates independently, identifying distinct categories of issues. This multi-perspective approach costs more tokens but produces dramatically higher code quality and security. High-stakes projects often use multiple independent LLM models for review (Codex, Claude, GPT), leveraging different model strengths.

Security Considerations in AI-Driven Development

Security in harness engineering requires deliberate architectural choices. Several fundamental patterns protect production systems:

Environment Variable Protection: Use hooks to completely block agents from accessing environment variable files. No amount of reasoning should allow credential exposure. Make this an absolute guardrail, not a suggestion.

Permission Levels: Codex offers multiple permission modes. "Basic" mode asks for confirmation on every potentially risky action (making development slow). "Automatic Review" mode (the recommended default) asks for confirmation only on flagged-as-risky actions, allowing routine work to proceed automatically. "Full Permission" mode should only be used in sandboxed environments.

Code Review as Security: Multi-stage code review catches security issues before production. Local review finds obvious problems. Pull request review catches sophisticated vulnerabilities. Human review validates overall architecture.

API Cost Controls: When integrating external APIs (like OpenAI), implement rate limiting and cost monitoring. Never expose API keys in code; always use environment variables and server-side calls. Consider quota limits per user to prevent abuse.

The honest assessment is that we're in a transitional period where perfect security is hard to achieve while maintaining developer velocity. The strategy is defense-in-depth: multiple overlapping safeguards, clear audit trails, and rapid incident response.

Common Patterns and Best Practices

Interview Before Building: Spending 30-60 minutes on the interview phase prevents dozens of hours of rework. Always conduct interviews for complex features, even if you think you know what you want.

Plan Documents Are Underrated: The investment in clear documentation pays dividends throughout development. It's your insurance against context window degradation, team misalignment, and scope creep.

Incremental Harness Building: Don't attempt perfection from the start. Add components when you identify the need: agent.md for rules, skills when you find redundancy, MCP when you need external systems, hooks for enforcement, sub-agents for parallelization.

Reuse Over Creation: Before building anything new, check if a skill already exists. Codex's official skills library includes image generation, OpenAI document reading, security checking, and more. Community skills (like Deep Interview) provide domain expertise instantly.

Context as a Limiting Resource: Treat context window like the precious resource it is. Progressive loading through skills, external documentation references, and memory activation all extend your effective context capacity.

Documentation in Multiple Formats: Keep plans in both Markdown (for AI processing) and HTML (for human review). The dual format accommodates both workflows seamlessly.

Advanced Integration: Codex Cloud and GitHub Automation

Codex Cloud (the web version) enables sophisticated integration with GitHub:

Code Review Automation: Enable code review in Codex Cloud Settings, connect your GitHub repositories, and define review triggers. The Codex bot automatically reviews every pull request, checking for security, code quality, and standard adherence.

Issue and PR Generation: Codex agents can create issues, branch automatically, make commits, and generate pull request descriptions directly from your conversation. This eliminates manual GitHub navigation while keeping all changes documented.

Deployment Tracking: Integration with Vercel, AWS, or other deployment platforms allows agents to autonomously handle deployment tasks, manage environment variables (securely), and verify successful deployments.

This automation transforms GitHub from a repository interface into an intelligent development system where agents handle routine tasks while humans focus on architectural decisions and strategic thinking.

The Broader Philosophy: Harness Engineering as Organizational Capability

Harness engineering extends beyond individual productivity—it becomes an organizational capability when team members share harness components. Your agent.md becomes team standards. Your skills become collective expertise. Your code review hooks ensure consistency across all projects.

Organizations using mature harness engineering see:

Faster onboarding: New developers inherit team standards through harness components instead of lengthy documentation
Fewer defects: Hooks and code review automation catch issues automatically
Knowledge accumulation: Skills and memory capture organizational expertise, preventing knowledge loss when team members leave
Consistent architecture: Shared harness components guide all projects toward coherent design patterns
Scalable expertise: Domain specialists create skills encoding their expertise, making that expertise available to the entire team

The investment in harness engineering is an investment in organizational capability and long-term productivity, not just individual speed.

Conclusion

Codex harness engineering represents a mature approach to AI-driven development that transforms unreliable prompting into production-ready systems. The six core components—agent.md, Memory, Skills, MCP, Hooks, and Sub-agents—provide a complete toolkit for designing, implementing, and controlling AI agent behavior.

The practical workflow (Interview → Planning → Setup → Development → Review → Deployment) ensures that projects remain aligned with original intent while adapting to discovered requirements. The Commit Hero example demonstrates this workflow end-to-end, from vague concept to deployed, AI-enhanced service.

Most importantly, harness engineering acknowledges a fundamental truth: AI agents work best within structured frameworks. Rather than relying on agents to reason their way to correct behavior (which is inconsistent), harness engineering enforces standards programmatically while enabling agent autonomy in appropriate domains.

For developers ready to move beyond prompt-based interaction toward production-grade AI-driven development, harness engineering provides the systematic approach that turns potential into reliability. Start with clear rules in agent.md, add skills for repetitive workflows, implement hooks for security, and progressively enhance your harness as projects grow. Your future self—and your team—will thank you for building systematic capability rather than relying on intuitive hoping.

원문출처: Codex 마스터 클래스 라이브 | 하네스 설계부터 서비스 배포까지 (라이브 마지막에는 작은 이벤트도 준비했습니다)

powered by osmu.app

Codex Harness Engineering: Complete Guide to AI Agent Development