self-improving-agent

SKILL.md

Self-Improving Agent

"An AI agent that learns from every interaction, accumulating patterns and insights to continuously improve its own capabilities." — Based on 2025 lifelong learning research

Overview

This is a universal self-improvement system that learns from ALL skill experiences, not just PRDs. It implements a complete feedback loop with:

Multi-Memory Architecture: Semantic + Episodic + Working memory

Self-Correction: Detects and fixes skill guidance errors

Self-Validation: Periodically verifies skill accuracy

Hooks Integration: Auto-triggers on skill events (before_start, after_complete, on_error)

Evolution Markers: Traceable changes with source attribution

Research-Based Design

Based on 2025 research:

Research
Key Insight
Application

SimpleMem
Efficient lifelong memory
Pattern accumulation system

Multi-Memory Survey
Semantic + Episodic memory
World knowledge + experiences

Lifelong Learning
Continuous task stream learning
Learn from every skill use

Evo-Memory
Test-time lifelong learning
Real-time adaptation

The Self-Improvement Loop

┌─────────────────────────────────────────────────────────────────┐
│                    UNIVERSAL SELF-IMPROVEMENT                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   Skill Event → Extract Experience → Abstract Pattern → Update  │
│        │                  │                │         │          │
│        ▼                  ▼                ▼         ▼          │
│   ┌─────────────────────────────────────────────────────┐       │
│   │              MULTI-MEMORY SYSTEM                      │       │
│   ├─────────────────────────────────────────────────────┤       │
│   │  Semantic Memory   │  Episodic Memory  │ Working Memory │  │
│   │  (Patterns/Rules)  │  (Experiences)    │  (Current)     │  │
│   │  memory/semantic/  │  memory/episodic/ │  memory/working/│  │
│   └─────────────────────────────────────────────────────┘       │
│                                                                 │
│   ┌─────────────────────────────────────────────────────┐       │
│   │              FEEDBACK LOOP                            │       │
│   │  User Feedback → Confidence Update → Pattern Adapt   │       │
│   └─────────────────────────────────────────────────────┘       │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

When This Activates

Automatic Triggers (via hooks)

Event
Trigger
Action

before_start
Any skill starts
Log session start

after_complete
Any skill completes
Extract patterns, update skills

on_error
Bash returns non-zero exit
Capture error context, trigger self-correction

Manual Triggers

User says "自我进化", "self-improve", "从经验中学习"

User says "分析今天的经验", "总结教训"

User asks to improve a specific skill

Evolution Priority Matrix

Trigger evolution when new reusable knowledge appears:

Trigger
Target Skill
Priority
Action

New PRD pattern discovered
prd-planner
High
Add to quality checklist

Architecture tradeoff clarified
architecting-solutions
High
Add to decision patterns

API design rule learned
api-designer
High
Update template

Debugging fix discovered
debugger
High
Add to anti-patterns

Review checklist gap
code-reviewer
High
Add checklist item

Perf/security insight
performance-engineer, security-auditor
High
Add to patterns

UI/UX spec issue
prd-planner, architecting-solutions
High
Add visual spec requirements

React/state pattern
debugger, refactoring-specialist
Medium
Add to patterns

Test strategy improvement
test-automator, qa-expert
Medium
Update approach

CI/deploy fix
deployment-engineer
Medium
Add to troubleshooting

Multi-Memory Architecture

1. Semantic Memory (memory/semantic-patterns.json)

Stores abstract patterns and rules reusable across contexts:

{
  "patterns": {
    "pattern_id": {
      "id": "pat-2025-01-11-001",
      "name": "Pattern Name",
      "source": "user_feedback|implementation_review|retrospective",
      "confidence": 0.95,
      "applications": 5,
      "created": "2025-01-11",
      "category": "prd_structure|react_patterns|async_patterns|...",
      "pattern": "One-line summary",
      "problem": "What problem does this solve?",
      "solution": { ... },
      "quality_rules": [ ... ],
      "target_skills": [ ... ]
    }
  }
}

2. Episodic Memory (memory/episodic/)

Stores specific experiences and what happened:

memory/episodic/
├── 2025/
│   ├── 2025-01-11-prd-creation.json
│   ├── 2025-01-11-debug-session.json
│   └── 2025-01-12-refactoring.json

{
  "id": "ep-2025-01-11-001",
  "timestamp": "2025-01-11T10:30:00Z",
  "skill": "debugger",
  "situation": "User reported data not refreshing after form submission",
  "root_cause": "Empty callback in onRefresh prop",
  "solution": "Implement actual refresh logic in callback",
  "lesson": "Always verify callbacks are not empty functions",
  "related_pattern": "callback_verification",
  "user_feedback": {
    "rating": 8,
    "comments": "This was exactly the issue"
  }
}

3. Working Memory (memory/working/)

Stores current session context:

memory/working/
├── current_session.json   # Active session data
├── last_error.json        # Error context for self-correction
└── session_end.json       # Session end marker

Self-Improvement Process

Phase 1: Experience Extraction

After any skill completes, extract:

What happened:
  skill_used: {which skill}
  task: {what was being done}
  outcome: {success|partial|failure}

Key Insights:
  what_went_well: [what worked]
  what_went_wrong: [what didn't work]
  root_cause: {underlying issue if applicable}

User Feedback:
  rating: {1-10 if provided}
  comments: {specific feedback}

Phase 2: Pattern Abstraction

Convert experiences to reusable patterns:

Concrete Experience
Abstract Pattern
Target Skill

"User forgot to save PRD notes"
"Always persist thinking to files"
prd-planner

"Code review missed SQL injection"
"Add security checklist item"
code-reviewer

"Callback was empty, didn't work"
"Verify callback implementations"
debugger

"Net APY position ambiguous"
"UI specs need exact relative positions"
prd-planner

Abstraction Rules:

If experience_repeats 3+ times:
  pattern_level: critical
  action: Add to skill's "Critical Mistakes" section

If solution_was_effective:
  pattern_level: best_practice
  action: Add to skill's "Best Practices" section

If user_rating >= 7:
  pattern_level: strength
  action: Reinforce this approach

If user_rating <= 4:
  pattern_level: weakness
  action: Add to "What to Avoid" section

Phase 3: Skill Updates

Update the appropriate skill files with evolution markers:

<!-- Evolution: 2025-01-12 | source: ep-2025-01-12-001 | skill: debugger -->

## Pattern Added (2025-01-12)

**Pattern**: Always verify callbacks are not empty functions

**Source**: Episode ep-2025-01-12-001

**Confidence**: 0.95

### Updated Checklist
- [ ] Verify all callbacks have implementations
- [ ] Test callback execution paths

Correction Markers (when fixing wrong guidance):

<!-- Correction: 2025-01-12 | was: "Use callback chain" | reason: caused stale refresh -->

## Corrected Guidance

Use direct state monitoring instead of callback chains:
```typescript
// ✅ Do: Direct state monitoring
const prevPendingCount = usePrevious(pendingCount);

### Phase 4: Memory Consolidation

1. **Update semantic memory** (`memory/semantic-patterns.json`)
2. **Store episodic memory** (`memory/episodic/YYYY-MM-DD-{skill}.json`)
3. **Update pattern confidence** based on applications/feedback
4. **Prune outdated patterns** (low confidence, no recent applications)

## Promotion Policy

Self-improvement has two separate jobs:

1. **Capture** facts, corrections, failed assumptions, and reusable patterns as memory or proposal artifacts.
2. **Promote** only validated patterns into `SKILL.md`, `AGENTS.md`, docs, or CLI behavior.

Default to capture-first. Promote a change only when one of these is true:

- The user explicitly asks to update a skill or repository instruction.
- The same pattern recurs across multiple episodes.
- A focused test or review proves the current guidance is wrong or incomplete.
- The change is low-risk documentation that preserves existing behavior and is clearly traceable.

Promotion targets:

| Artifact | Use For | Approval Level |
|----------|---------|----------------|
| `memory/episodic/*.json` | Raw episode facts and signals | Auto |
| `memory/semantic-patterns.json` | Candidate reusable patterns with confidence | Auto |
| `memory/proposals/*.md` | Proposed skill/doc/code changes with evidence | Auto |
| `SKILL.md` / `references/` | Validated workflow guidance | Ask first unless user requested editing |
| `AGENTS.md` / repo rules | Cross-repo behavior or hard constraints | Ask first |
| CLI/runtime code | Automation semantics | Require tests |

## Self-Correction (on_error hook)

Triggered when:
- Bash command returns non-zero exit code
- Tests fail after following skill guidance
- User reports the guidance produced incorrect results

**Process:**

```markdown
## Self-Correction Workflow

1. Detect Error
   - Capture error context from working/last_error.json
   - Identify which skill guidance was followed

2. Verify Root Cause
   - Was the skill guidance incorrect?
   - Was the guidance misinterpreted?
   - Was the guidance incomplete?

3. Create Proposal
   - Write a proposal with evidence, affected skill names, and expected behavior
   - Add correction marker text in the proposal, not directly in the skill yet
   - Update related patterns in semantic memory with low initial confidence

4. Validate Fix
   - Test the corrected guidance
   - Ask user to verify

5. Promote
   - Apply the skill/doc/code change after validation or explicit approval
   - Keep the source episode/proposal id in the change note

Example:

<!-- Correction: 2025-01-12 | was: "useMemo for claimable ids" | reason: stale data at click time -->

## Self-Correction: Click-Time Computation

**Issue**: Using useMemo for claimable IDs caused stale data
**Fix**: Compute at click time for always-fresh data
**Pattern**: click_time_vs_open_time_computation

Self-Validation

Use the validation template in references/appendix.md when reviewing updates.

Hooks Integration

Runtime Trigger Source

agent-playbook self-improve reads skill chaining from each skill's SKILL.md frontmatter:

metadata:
  hooks:
    after_complete:
      - trigger: self-improving-agent
        mode: background
        reason: "Extract patterns"

Treat metadata.hooks as the source of truth. Do not maintain a second hardcoded hook map in runtime code. This keeps skill behavior auditable and lets Skill Creator style reviews inspect the same file that the agent executes.

Wiring Hooks in Claude Code Settings

For Claude Code, install hooks through agent-playbook init --hooks when possible.
If you need manual setup, add hook entries to Claude Code settings at the
appropriate user or project scope.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|Write|Edit",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/pre-tool.sh \"$TOOL_NAME\" \"$TOOL_INPUT\""
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/post-bash.sh \"$TOOL_OUTPUT\" \"$EXIT_CODE\""
          }
        ]
      }
    ],
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "bash ${SKILLS_DIR}/self-improving-agent/hooks/session-end.sh"
          }
        ]
      }
    ]
  }
}

Replace ${SKILLS_DIR} with your actual skills path.

Additional References

See references/appendix.md for memory structure, workflow diagrams, metrics, feedback templates, and research links.

Best Practices

DO

✅ Learn from EVERY skill interaction

✅ Extract patterns at the right abstraction level

✅ Update multiple related skills

✅ Track confidence and apply counts

✅ Ask for user feedback on improvements

✅ Use evolution/correction markers for traceability

✅ Validate guidance before applying broadly

✅ Write proposals before mutating durable skill guidance

✅ Keep hook routing in metadata.hooks

DON'T

❌ Over-generalize from single experiences

❌ Update skills without confidence tracking

❌ Ignore negative feedback

❌ Make changes that break existing functionality

❌ Create contradictory patterns

❌ Update skills without understanding context

❌ Silently promote self-improvement findings into repo rules

❌ Duplicate hook definitions in CLI code and skill frontmatter

Quick Start

After a high-signal skill workflow completes, this agent can:

Analyzes what happened

Extracts patterns and insights

Writes memory and proposal artifacts

Promotes validated improvements only when approval or evidence is sufficient

Reports summary to user

References

SimpleMem: Efficient Lifelong Memory for LLM Agents

A Survey on the Memory Mechanism of Large Language Model Agents

Lifelong Learning of LLM based Agents

Evo-Memory: DeepMind's Benchmark

Let's Build a Self-Improving AI Agent

OpenCrabs local self-improving agent

ELL-StuLife experience-driven lifelong learning