Intrinsic curiosity-driven continuous learning: detect gaps between expected and actual results, treat them as curiosity signals, and update skills according...
---
name: curiosity-loop
description: "Intrinsic curiosity-driven continuous learning: detect gaps between expected and actual results, treat them as curiosity signals, and update skills accordingly. Inspired by developmental AI from Flowers Lab, INRIA."
version: 1.0.0
author: Guillaume D
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [learning, self-improvement, curiosity, developmental-AI, delta-tracking, continuous-improvement]
related_skills: [blogwatcher, arxiv, polymarket]
---
# Curiosity Loop — Intrinsic Curiosity-Driven Continuous Learning
## Overview
A self-improvement framework that transforms knowledge gaps and failed outcomes into structured learning cycles. Inspired by developmental AI research from [Flowers Lab, INRIA](https://www.robots.org/flowers-lab/) and the work of Cédric Colas.
Rather than treating failures as errors, the Curiosity Loop treats them as **curiosity signals** — prompts for active exploration, skill updates, and curriculum building. This creates a continuous self-improvement loop that makes the agent progressively better at its domain.
### Core Concepts (from developmental AI)
| Concept | Source | Application |
|---------|--------|-------------|
| **Automatic Curriculum Learning (ACL)** | Portolas et al. (Flowers Lab) | Structured progression, not random learning |
| **Autotelic Activity** | Vygotsky / Colas | Learning for its own sake — intrinsic motivation |
| **Zone of Proximal Development (ZPD)** | Vygotsky | Learning "just beyond" current capability |
| **Map-Elites / Behavioral Diversity** | Mouret & Clune | Maximize repertoire diversity, not single optimization |
| **Semantic Interference** | Flowers Lab (CLIP research) | Biases reveal representation limits |
## When to Activate the Curiosity Loop
### Activation Signals (prioritize 1 at a time)
1. **Result delta**: A response/action does not produce the expected effect
2. **Iterative loop**: User repeats the same correction 2+ times
3. **Incomplete knowledge**: Detecting missing information for a correct answer
4. **Sub-optimal approach**: Recognizing a better approach exists
5. **Knowledge scan**: Discovering new concepts/tools via periodic scanning
### Do NOT activate when
- The result is correct and satisfactory (no delta)
- The user has not expressed a need for improvement
- The context is trivial (not worth the learning cycle)
## The Loop — 6 Numbered Steps
### Step 1: ACTION
Attempt to solve the problem with current knowledge.
### Step 2: DELTA
Identify the precise gap between expected and actual results.
- What did not work as expected?
- What was the expectation vs. reality?
- How much extra time/iterations did this cost?
### Step 3: CURIOSITY
Treat the delta as a learning signal, not a failure.
- What did I not know or not know well?
- What concept/tool/pattern was missing?
- Why was this information important?
### Step 4: RESEARCH
Actively explore the missing concept.
- Search docs, existing skills, external sources
- Use web_search, browser, terminal, session_search
- Validate the discovery's relevance
### Step 5: INTEGRATION
Update knowledge/skills.
- **If an existing skill is incomplete**: `skill_manage(action='patch')` with new info
- **If a new pattern is discovered**: `skill_manage(action='create')` for a new skill
- **If a stable fact is learned**: `memory(action='add')` for durable facts
- **If a tool/command is discovered**: document in the appropriate skill
### Step 6: CURRICULUM
Structure learning as progressive milestones.
- Identify missing prerequisites
- Decompose the concept into progressive sub-concepts
- Create verifiable milestones (can I explain this? can I use it?)
## Delta Tracking
Track all deltas in `~/.hermes/deltas.json` for auditability and periodic review.
Format:
```json
{
"deltas": [
{
"id": "delta-001",
"date": "2026-05-22",
"context": "Short description of the context",
"expected": "What was expected",
"actual": "What happened",
"gap": "What was missing (concept/tool/pattern)",
"resolution": "How the gap was filled",
"skill_updated": "Name of updated skill, or null",
"status": "resolved | open | learning"
}
]
}
```
## Diversity Policies (Map-Elites)
### Diversity Rule
When finding an effective pattern for a problem type, also:
1. Document the pattern in a skill
2. Identify 1-2 alternative approaches (even if less optimal)
3. Note cases where the alternative would be preferable
### Periodic Diversity Scan
Every ~30 days, check:
- Which patterns do I rarely or never use?
- Are there domains where I have only one approach?
- What skills could be enriched?
## Concrete Examples
### Example 1: Browser Playwright broken on NixOS
- **Delta**: The browser tool did not work (missing shared libraries)
- **Curiosity**: Why? What's the alternative?
- **Research**: Discovered Obscura as a headless browser alternative
- **Integration**: Created `obscura-browser` skill, saved fact to memory
- **Curriculum**: Learn Obscura commands in order: fetch → serve → scrape → stop
### Example 2: himalaya installed without nix-shell
- **Delta**: himalaya worked directly without nix-shell
- **Curiosity**: How is it installed? Where?
- **Research**: Verified binary location and config
- **Integration**: Saved to memory: "himalaya installed directly, no nix-shell needed"
### Example 3: Command validation with tirith
- **Delta**: Needed to validate shell commands before execution
- **Curiosity**: What tool validates command safety?
- **Research**: Discovered `tirith` and `vet` for command validation
- **Integration**: Note in memory to use validation tools for shell commands
## Periodic Knowledge Scanning
The Curiosity Loop includes a built-in mechanism for proactive knowledge discovery. Configure sources in `~/.hermes/deltas.json`:
```json
{
"scan_sources": [
{
"name": "Flowers INRIA",
"type": "youtube_channel",
"url": "https://www.youtube.com/channel/UCrBNVs3u3mwlRsm2v3EKuXA",
"last_scanned": "2026-05-22"
}
]
}
```
Run the scanner:
```bash
python3 ~/.hermes/skills/research/curiosity-loop/scripts/scan_sources.py
```
Or schedule it as a cron job (runs silently if nothing new):
```
0 9 * * 0 → Sunday 9am weekly
```
## Maintenance
### When to patch this skill
- If loop steps become redundant or obsolete
- If new activation signals are discovered
- If the delta tracking format changes
### When to create a new skill
- If a discovered concept deserves its own documentation
- If a repeated usage pattern (>3 times) is identified
- If a tool/new technology gains importance
## Quality Criteria
A delta is well-treated when:
- [ ] The gap is precisely identified (not vague)
- [ ] Research was actually performed (not just assumed)
- [ ] Integration is durable (skill/memory, not just conversation)
- [ ] Curriculum is structured (milestones, not just "I learned it")
- [ ] Diversity is maintained (at least one alternative documented)
## References
- **Automatic Curriculum Learning for Deep RL**: Portolas et al., Flowers Lab + Microsoft Research + OpenAI
- **Automatic Curriculum Learning for Developmental Machine Learners**: Cédric Colas, INRIA PhD Thesis (2022)
- **Autotelic Agents**: Colas et al., Flowers Lab — intrinsic motivation for language acquisition
- **Map-Elites**: Mouret & Clune, "Illuminating Search Spaces" (2015)
- **Semantic Interference in CLIP**: Flowers Lab research on picture-word interference in multimodal models
---
*Inspired by developmental AI research at Flowers Lab, INRIA. Published under MIT License.*
don't have the plugin yet? install it then click "run inline in claude" again.
the curiosity loop transforms knowledge gaps and failed outcomes into structured learning cycles. instead of treating failures as errors, it treats them as curiosity signals: prompts for active exploration, skill updates, and curriculum building. use this skill when you encounter unexpected results, repeated corrections, missing knowledge, or suboptimal approaches. it creates a continuous self-improvement loop that makes you progressively better at your domain, grounded in developmental AI research from Flowers Lab, INRIA.
external connections:
skill_manage: API or internal function to create/patch skills. requires write access to skill repository (typically ~/.hermes/skills/)memory: storage API for durable facts (typically ~/.hermes/memory.json or equivalent)web_search: external search capability (curl, requests library, or API wrapper)browser: headless browser tool (Playwright, Puppeteer, or Obscura as fallback)session_search: internal session/conversation history searchterminal: shell execution for tool discovery and validationcontext required:
~/.hermes/deltas.json), must exist or be initialized on first runoptional env vars:
HERMES_DELTA_PATH: override default deltas.json location (default: ~/.hermes/deltas.json)HERMES_SKILL_REPO: override skill repository path (default: ~/.hermes/skills/)HERMES_MEMORY_PATH: override memory storage location (default: ~/.hermes/memory.json)step 1: action
step 2: delta
step 3: curiosity
step 4: research
step 5: integration
skill_manage(action='patch', skill_name=X, updates=...) with new infoskill_manage(action='create', skill_name=new_name, content=...) for new skillmemory(action='add', fact=X, category=Y, confidence_level=Z) for durable knowledgestep 6: curriculum
delta logging (after all steps):
~/.hermes/deltas.json with id, date, context, expected, actual, gap, resolution, skill_updated, statusactivation: activate curiosity loop if and only if:
do NOT activate when:
during research (step 4):
during integration (step 5):
~/.hermes/deltas_pending.jsonedge cases:
success state:
~/.hermes/deltas.json with complete fields: id, date, context, expected, actual, gap, resolution, skill_updated (or null), status~/.hermes/research_reports/<delta_id>.md~/.hermes/skills/<skill_name>/curriculum.mdfile locations:
~/.hermes/deltas.json~/.hermes/deltas_pending.json~/.hermes/research_reports/<delta_id>.md~/.hermes/skills/<skill_name>/curriculum.md~/.hermes/deltas.json under top-level scan_sources array~/.hermes/skills/research/curiosity-loop/scripts/scan_sources.pydelta.json schema:
{
"deltas": [
{
"id": "delta-YYYYMMDD-NNN",
"date": "2026-05-22T14:30:00Z",
"context": "short description of the context",
"expected": "what was expected",
"actual": "what happened",
"gap": "concept/tool/pattern/fact that was missing",
"resolution": "how the gap was filled or why it remains open",
"skill_updated": "skill_name_slug or null",
"status": "resolved | learning | open | blocked_auth | integration_failed"
}
],
"scan_sources": [
{
"name": "source name",
"type": "youtube_channel | github_repo | arxiv_keyword | web_url",
"url": "https://...",
"last_scanned": "2026-05-22T09:00:00Z",
"scan_interval_days": 30
}
]
}
the skill worked if:
~/.hermes/deltas.json with status = "resolved"~/.hermes/research_reports/<delta_id>.md with sources and findings~/.hermes/skills/<skill_name>/ and see your changes integrated