Agent Harness

A unified engineering harness combining execution discipline, knowledge compounding, and product thinking. Born from ~450k characters of real-world AI textbook writing and 15+ production incidents.

GAIA benchmark shows scaffold design = 30pp+ performance boost. Same model, HAL scaffold 74.6% vs bare model ~44%. The harness is the multiplier.

intent

Use this skill when building features, fixing bugs, planning specs, reviewing code, shipping changes, or debugging with AI agents. Triggers on "agent harness", "engineering workflow", "build protocol", "multi-agent task", "coding discipline", or "subagent orchestration". The harness wraps three layers (Challenge, Execute, Compound) around agent work to enforce correctness, prevent rationalization, and extract lessons from every task. Run this skill whenever you need structured engineering discipline with quality gates, especially for multi-step or multi-agent tasks.

inputs

Agent execution environment:

• Current context window percentage (auto-tracked by platform)
• Subagent availability (subagents list returns count and status)
• File system access (read, write, append)
• Execution tools (shell, read, write, append, ls, wc, grep)
• External APIs (Bedrock, GitHub, etc. if needed for the task)

Task inputs:

• User request (build, plan, spec, review, ship, debug command)
• Existing code or files (path to repo, config, docs)
• Acceptance criteria (if provided; harness will infer if missing)
• Constraints (budget, timeline, risk tolerance, rollback window)

External connections (if applicable):

• Bedrock API (env var: AWS_BEDROCK_REGION, AWS_BEDROCK_MODEL_ID; rate limit: 4 concurrent requests max, use model rotation or serial fallback)
• GitHub (env var: GITHUB_TOKEN, scope: repo, read:org; auth expires, check before large operations)
• Monitoring/Logging (if shipping to prod, verify endpoint and credentials before merge)

Context budget:

• Current token usage (tracked before each tool call)
• Observation masking rules (defined in ## context budget management below)

procedure

Step 1: Auto-grade task complexity (all tasks)

Input: User request + existing code (if any)

Output: Complexity level (🟢 simple, 🟡 medium, 🔴 complex) + reasoning

How to grade:

• 🟢 Simple: bug fix, config change, small tweak. Single file, no architecture impact. ~1 file, <50 lines changed.
• 🟡 Medium: new feature, new module, integration. Spans 2-5 files, clear API contract, no system redesign.
• 🔴 Complex: architecture change, multi-module redesign, new system. Spans 5+ files, unknown unknowns, refactors existing abstractions.

Decision: If unsure, pick 🟡. Upgrade to 🔴 during execution if you discover hidden dependencies. Never downgrade.

Next step: Go to Step 2.

---

Step 2: Run Layer 1 (Challenge) if 🔴 complex (conditional)

Input: Task description, problem statement

Output: One-paragraph problem statement confirmed by user

Do this only if complexity = 🔴:

Answer these questions before any code:

1. Problem validity , Is the user solving a real problem? (Not a nice-to-have proxy?)
2. Simplest approach , Is there a simpler way to solve this?
3. Scope clarity , Can you explain "done" in one sentence? (If no, scope is fuzzy.)
4. Risk assessment , What's the worst outcome if this goes wrong? (Rollback time? Data loss? User impact?)

Write a one-paragraph problem statement. Show it to the user. Wait for confirmation.

If user rejects: Stop, don't proceed. Refine the problem statement and re-submit.

If user confirms: Go to Step 3.

If complexity is 🟢 or 🟡: Skip this step, go to Step 3.

---

Step 3: Run Layer 2 (Execute) , Spec phase (if 🟡 or 🔴)

Input: Complexity level, problem statement (if 🔴), user request

Output: Spec document (written to spec.md or inline if <200 words)

Do this for 🟡 and 🔴:

Write a spec with four sections:

1. Goal , One sentence describing the outcome.
2. Interface , Inputs, outputs, API contracts. What calls what? What returns what?
3. Constraints , What you will NOT do. (Explicit scope boundaries prevent creep.)
4. Acceptance criteria , How to verify it works. Must be testable (run a command, check output, assert a property).

Show the spec to the user before proceeding.

If 🟢: Skip spec, go to Step 4 (Plan).

Next step: Go to Step 4.

---

Step 4: Run Layer 2 (Execute) , Plan phase (if 🟡 or 🔴)

Input: Spec, existing codebase

Output: Task list (written to plan.md)

Do this for 🟡 and 🔴:

Break the spec into atomic tasks:

• Each task modifies ≤3 files.
• Each task has a clear, runnable verification step (not "review the code").
• Tasks ordered by dependency. Independent tasks can run in parallel (future: subagent delegation).

Example format:

Task 1: [File] Create config schema
  Changes: new file `config.schema.json`
  Verify: `jq . < config.schema.json` (valid JSON)
  
Task 2: [Feature] Load config on startup
  Changes: `app.py` lines 12-18
  Verify: `python -m pytest tests/test_config.py -v` (pass)
  Depends on: Task 1

Show the plan to the user. Refine if needed.

If 🟢: Skip plan, go to Step 5 (Build).

Next step: Go to Step 5.

---

Step 5: Build (all tasks)

Input: Plan (or task description if 🟢), codebase, spec (if written)

Output: Code changes, config changes, generated files

Critical rules:

• Never modify code you haven't read first.
• Don't add features beyond what was asked.
• Don't refactor "while you're at it" (file a TODO instead).
• If tests fail, report honestly. Don't claim success.

For each task in the plan (or the single 🟢 task):

1. Read the files you'll modify (understand the code, don't blindly edit).
2. Make the change (edit, create, or append).
3. Run the verification step immediately (see Step 6 below).
4. Checkpoint progress to file (append to progress.md: "Task N complete. Changed files: X, Y, Z. Status: PASS.").
5. Only then move to the next task.

If a task fails verification: Debug, fix, re-verify. Only mark PASS when verification succeeds.

Next step: Go to Step 6.

---

Step 6: Verify every deliverable (all tasks)

Input: Code change, config change, file, API response, or doc

Output: Evidence (test output, command output, grep results, response JSON)

Core rule: Reading is not verification. Run it.

Deliverable type	Required evidence
Code change	Tests pass (show output, pytest -v or equivalent)
Config change	Restart + verify (show status, systemctl status or equivalent)
File generation	Line count + grep key content (wc -l, grep "keyword")
API integration	Show actual response (curl output, JSON, status code)
Documentation	Spot-check 3 claims for accuracy (e.g., "does the code actually do what the doc says?")

For each deliverable:

1. Run the verification step from the plan (or define one if 🟢).
2. Capture the output (copy-paste or screenshot).
3. Verify the output matches the acceptance criteria.
4. If no match: debug and re-run, don't move on.
5. If match: record the evidence in progress file and move to the next task.

Next step: If all tasks pass verification, go to Step 7. If any fail, debug and re-run Step 5-6 for that task.

---

Step 7: Review (if 🟡 or 🔴)

Input: All code changes, all verification output

Output: Review summary (written to review.md)

Do this for 🟡 and 🔴:

Self-review from 5 dimensions:

1. Correctness , Does it do what was asked? Check against spec + acceptance criteria.
2. Edge cases , Empty input, huge input, concurrent access, network timeout, auth expiry? Can you think of 3 ways it could break?
3. Security , Injection points, leaked secrets, missing auth, overpermissioned API keys, SQL injection, unvalidated input?
4. Performance , Will it work at 10x scale? 100x? Any N+1 queries, infinite loops, unbounded memory?
5. Maintainability , Will someone understand this in 6 months? Are there comments on non-obvious logic? Can you find the code by searching a keyword?

For each dimension, pick one issue if you find one. Write it down with a fix (don't leave "TODO: think about this").

If 🟢: Skip review, go to Step 8.

Next step: Go to Step 8.

---

Step 8: Ship (if 🔴 only)

Input: All code changes, all verification output, review summary

Output: Deployment confirmation + rollback confirmation

Do this for 🔴 only:

Pre-ship checklist:

• All tests pass (show output from Step 6).
• Rollback plan exists (undo in <5 min? How? Write it down.).
• Feature flag or gradual rollout if risky (new API endpoint? New database schema? Gate it.).
• Monitoring covers the new code path (log key events, set up alerts).

After checks pass:

1. Run the deploy command or merge to main.
2. Verify the deployment succeeded (check logs, health check, or smoke test).
3. Record the deploy time and status in progress file.

If 🟢 or 🟡: Skip ship, go to Step 9.

Next step: Go to Step 9.

---

Step 9: Compound (all tasks)

Input: What happened during the task (errors, retries, surprises, bottlenecks)

Output: Lesson recorded in lessons.md

After completing any task, spend 30 seconds on:

1. What broke? , Errors, retries, unexpected behavior? Record the specific lesson.
2. What was slow? , Bottlenecks? Note them (e.g., "Bedrock throttles at >4 concurrent requests. Use model rotation or serial execution.").
3. What would you do differently? , Better approach with hindsight?

Record only specific, actionable lessons.

Good: "Bedrock throttles at >4 concurrent requests. Use model rotation or serial execution."

Bad: "Remember to handle API limits properly."

Write to lessons.md. If this is a lesson worth sharing, suggest it as a future skill refinement.

Task complete. Go to outcome signal section below.

---

decision points

If complexity is 🟢 (simple):

• Skip Challenge (Step 2), Spec (Step 3), Plan (Step 4), Review (Step 7), Ship (Step 8).
• Run only: Build (Step 5), Verify (Step 6), Compound (Step 9).
• Time to completion: <10 minutes.

If complexity is 🟡 (medium):

• Skip Challenge (Step 2), Ship (Step 8).
• Run: Spec (Step 3), Plan (Step 4), Build (Step 5), Verify (Step 6), Review (Step 7), Compound (Step 9).
• Time to completion: 30 minutes to 2 hours.

If complexity is 🔴 (complex):

• Run all steps: Challenge (Step 2) through Ship (Step 8), plus Compound (Step 9).
• Time to completion: 2+ hours (depends on scope).

If user rejects the problem statement (Challenge step):

• Stop. Do not proceed to Spec.
• Refine the problem statement with the user.
• Re-submit for confirmation.

If a verification step fails:

• Debug the failure (re-read code, check logs, test locally).
• Fix the code or test.
• Re-run the verification step.
• Only mark the task PASS if verification succeeds.
• Do not proceed to the next task until current task verifies.

If context budget exceeds 85%:

• Activate Auto-Concise mode (Step within "Context Budget Management" below).
• No new large files, truncate tool output, delegate reads to subagent if available.

If context budget exceeds 95%:

• Activate Emergency mode.
• Flush state to file, alert user to /reset, stop accepting new tasks.

If subagent count is at or near capacity (≤4 hard limit):

• Check subagents list before spawning new subagents.
• If 4 subagents already running, re-slice planned work into sequential batches.
• Do not spawn the 5th subagent.

If a task requires reading a large file and context is tight (70-85%):

• Use grep and wc -l instead of read to extract key info.
• Checkpoint findings to file.
• Clear observations from context (observation masking).

If tool call has no follow-up action:

• Danger signal. Stop and pick an action immediately.
• Never respond with "I'll continue..." without a next tool call.

---

output contract

Deliverables by complexity level:

🟢 Simple

• Single file with change (or new file created).
• Verification output (test pass, status check, or grep result).
• progress.md with one entry: "Task complete. Changed files: X. Status: PASS."
• lessons.md if any surprises (optional).

🟡 Medium

• spec.md , Goal, Interface, Constraints, Acceptance criteria.
• plan.md , Numbered tasks, dependencies, verification steps.
• Changed files (2-5 files modified or created).
• Verification output for each task (tests pass, curl output, etc.).
• progress.md , One line per task: "Task N: [name]. Changed files: X, Y. Status: PASS."
• review.md , 5-dimension review summary (correctness, edge cases, security, performance, maintainability).
• lessons.md , Specific, actionable lessons.

🔴 Complex

• challenge.md , Problem statement (one paragraph).
• spec.md , Goal, Interface, Constraints, Acceptance criteria.
• plan.md , Numbered tasks with dependencies, verification steps.
• Changed files (5+ files modified or created).
• Verification output for each task (tests, curl, logs, grep results).
• progress.md , One line per task.
• review.md , 5-dimension review.
• Deployment confirmation (log output, timestamp, rollback plan).
• lessons.md , Specific, actionable lessons.

File locations:

• All files written to current working directory or task-specific folder.
• Use ls to confirm files exist.
• Use wc -l and grep to spot-check content.

Data formats:

• Markdown for docs (*.md).
• JSON for API responses (verified with jq).
• Shell output for command verification (logged as-is).
• Test output for code verification (pytest, unittest, or equivalent runner output).

---

outcome signal

Task is complete when:

1. Complexity correctly assessed , User agrees on 🟢, 🟡, or 🔴 level (implicit for 🟢).
2. Problem statement confirmed (🔴 only) , User says "yes, that's the problem."
3. Spec confirmed (🟡🔴 only) , User says "yes, this is what we're building."
4. All tasks verified , Every task shows evidence (test pass, command output, grep result, API response).
5. Review complete (🟡🔴 only) , Review summary filed with no blocking issues.
6. Deployment confirmed (🔴 only) , Deploy logs show success, rollback plan documented.
7. Lessons recorded , At least one specific lesson in lessons.md (or "no surprises" if genuinely clean run).

User-facing signals:

• All task verification output is visible and matches acceptance criteria.
• progress.md shows all tasks with PASS status.
• No failing tests, no pending TODOs, no "I'll handle this later" statements.
• If shipping (🔴), deploy confirmation logged with timestamp and rollback plan.

Failure signal:

• Any task verification fails and is not debugged/fixed before marking complete.
• Tests fail and errors are suppressed or ignored.
• Spec or plan was skipped when it should have been written.
• Review raised an issue and it was not fixed.

---

anti-rationalization table

Reference this table whenever you feel the urge to skip a step or declare success without evidence.

Your excuse	Why it's wrong	Do this instead
"Too simple to need tests"	40% of P0 incidents come from "too simple" code.	Write the test. It takes 2 minutes.
"I already checked, looks fine"	Reading is not verifying.	Run it. ls, wc -l, grep, actual execution.