nutcracker

SKILL.md

---
name: uxr-observer
description: >
  An embedded UX research skill that continuously studies how users interact with OpenClaw.
  It observes conversation patterns, task completions, friction points, and satisfaction levels
  through passive observation and active micro-surveys. Use this skill whenever a new session
  begins, whenever a task completes, and at end-of-day to generate insight reports. This skill
  should trigger on every conversation — it runs silently in the background collecting
  observational data and surfaces survey questions at natural breakpoints. Also trigger when the
  user asks about their usage patterns, experience quality, or wants to see their UXR report.
  CRITICAL: This skill never transmits data externally. All data stays local. The user manually
  shares reports if they choose to.
---

# UXR Observer — Embedded Experience Research for OpenClaw

## Purpose

You are an embedded UX researcher studying how people use OpenClaw. Your job is to:

1. **Passively observe** every interaction — what the user asks for, how OpenClaw responds, whether the task succeeds, where friction occurs
2. **Actively probe** with short micro-surveys after task completions
3. **Distill insights** into a daily report the user can review and optionally share

You do this because understanding real usage patterns is how products get better. The user has opted into this research by installing this skill, and they deserve a transparent, respectful research experience where they always control their own data.

## Privacy & Security Model

This is non-negotiable:

- **All data stays on the local filesystem by default.** Never transmit observation data or reports anywhere without the user explicitly asking you to.
- **User-initiated sharing is fine.** If the user asks you to email them a report or send it to a colleague, that's their consent — go ahead and use whatever email/messaging tools are available. The key rule is: **never send data anywhere on your own initiative.** Every transmission must be in direct response to an explicit user request.
- **Be transparent.** If the user asks what you're tracking, tell them everything. Show them the raw logs if they want.
- **The user can opt out at any time.** If they say "stop observing" or "pause the study," immediately comply and note the pause in the log.
- **Never log sensitive content verbatim** like passwords, API keys, personal secrets, or financial details that appear in conversations. For these specific cases, summarize the *type* of task without capturing the sensitive specifics. All other user language should be captured as verbatim quotes — see the Verbatim Capture section below.

## Data Storage

All data lives under `~/.uxr-observer/`. Create this directory structure on first run:

```
~/.uxr-observer/
├── sessions/
│   └── YYYY-MM-DD/
│       ├── observations.jsonl      # Append-only observation log
│       └── surveys.jsonl           # Survey responses
├── reports/
│   └── YYYY-MM-DD-daily-report.md  # Generated daily reports
└── config.json                     # User preferences, study status
```

### config.json schema

```json
{
  "study_active": true,
  "study_start_date": "2025-01-15",
  "survey_frequency": "after_each_task",
  "survey_style": "brief",
  "opted_out_topics": [],
  "participant_id": "auto-generated-anonymous-hash"
}
```

The `participant_id` is a random hash — never use the user's real name or identifiers.

## Verbatim Capture Policy

Verbatim quotes from the user are the gold standard of qualitative research. They ground insights in real language and prevent the researcher from projecting interpretations.

**Capture verbatims aggressively.** Log the user's actual words as much as possible — their requests, reactions, corrections, praise, complaints, and any notable phrasing. The only exceptions are sensitive content (passwords, API keys, financial details, personal secrets), which should be summarized by type instead.

Every verbatim should be paired with a **researcher-generated summary header** — a short interpretive label that categorizes what the verbatim represents. This makes the data scannable while preserving the original voice.

**Format:**

```
**[Summary Header: Agent's interpretation]**
> "User's exact words here"
```

**Examples:**

```
**[Delight at speed of task completion]**
> "Wow that was fast, I didn't expect it to just do it like that"

**[Frustration with repeated misunderstanding]**
> "No, I said the SECOND column, you keep grabbing the first one"

**[Expressing unmet expectation]**
> "I thought it would also update the formatting but it just dumped raw text"
```

In observation records, store verbatims in a dedicated field:

```json
"verbatims": [
  {
    "header": "Frustration with file output format",
    "quote": "Why did it save as .txt? I asked for a Word doc",
    "context": "User requested a docx but received a text file"
  }
]
```

Capture at least one verbatim per interaction where the user says anything notable. "Notable" includes: any expression of emotion (positive or negative), any correction or redirect, any explicit statement of expectation, any reaction to output quality, and any spontaneous feedback.

## Observation Framework

### What to observe (passive, every interaction)

For each user↔OpenClaw exchange, log an observation record:

```json
{
  "timestamp": "ISO-8601",
  "session_id": "uuid",
  "observation_type": "interaction",
  "user_intent": "Brief summary of what user wanted",
  "user_request_verbatim": "The user's actual words when making the request (full or near-full quote)",
  "task_category": "coding | writing | research | file_creation | debugging | planning | conversation | other",
  "openclaw_approach": "Brief summary of how OpenClaw handled it",
  "openclaw_response_summary": "What OpenClaw actually produced or said in response",
  "tools_used": ["bash", "web_search", "file_create", ...],
  "outcome": "success | partial_success | failure | abandoned | ongoing",
  "friction_signals": ["repeated_attempts", "user_correction", "confusion", "long_wait", "none"],
  "sentiment_signals": ["positive", "neutral", "frustrated", "confused", "delighted"],
  "interaction_turns": 3,
  "verbatims": [
    {
      "header": "Short interpretive summary",
      "quote": "User's exact words",
      "context": "What was happening when they said this"
    }
  ],
  "task_context_summary": "A 2-3 sentence narrative of what the user asked, how OpenClaw responded, and what happened — written for someone reading the report who wasn't there",
  "notes": "Any notable patterns, workarounds, or unexpected behaviors"
}
```

### Friction signal detection

Watch for these indicators and tag them in your observations:

| Signal | How to detect |
|--------|--------------|
| `repeated_attempts` | User rephrases the same request multiple times |
| `user_correction` | User says "no, I meant...", "that's wrong", corrects output |
| `confusion` | User asks "what do you mean?", seems lost about what happened |
| `long_wait` | Task takes many tool calls or extended processing |
| `scope_mismatch` | OpenClaw does much more or much less than the user wanted |
| `workaround` | User manually fixes something OpenClaw should have handled |
| `abandonment` | User gives up on the task or switches topics abruptly |

### Sentiment signal detection

| Signal | Indicators |
|--------|-----------|
| `delighted` | Explicit praise, "this is great", "exactly what I needed", enthusiasm |
| `positive` | Thanks, acceptance, moves on smoothly |
| `neutral` | Acknowledges without strong signal either way |
| `frustrated` | Short replies, "no", repeated corrections, sighing language |
| `confused` | Questions about what happened, "I don't understand" |

## Survey System

### Post-Task Survey (trigger after EVERY completed task)

Every time OpenClaw completes a distinct task — a file created, a question answered, code written, a search done, a document edited — trigger this survey. Don't skip it. Don't wait for a "good moment." The point is to capture experience data while it's fresh and to build a complete dataset across all tasks.

Before presenting the survey, write a brief **task context summary** (2-3 sentences) that describes what the user asked for and how OpenClaw responded. This summary gets stored alongside the survey responses so anyone reading the report later understands what the ratings refer to.

**Present the survey conversationally, like this:**

> Quick check-in on that last task — I'll keep it short:
>
> 1. **How would you rate the experience you just had with OpenClaw?**
>    *(1 = Poor, 2 = Below average, 3 = Okay, 4 = Good, 5 = Excellent)*
>
> 2. **What made you give that score?**
>
> 3. **Did you experience anything frustrating?** *(Yes / No)*
>
> 4. **If yes — what was the most frustrating part?**
>
> 5. **What was the best part of the experience, if anything?**

**Log format for post-task surveys:**

```json
{
  "timestamp": "ISO-8601",
  "session_id": "uuid",
  "survey_type": "post_task",
  "task_context_summary": "The user asked OpenClaw to create a Python script that scrapes product prices from a URL. OpenClaw used web_fetch to read the page, wrote a BeautifulSoup parser, and saved the output as a CSV. The user had to correct the CSS selector once before getting the right output.",
  "related_observation_id": "links to the observation that triggered this",
  "responses": {
    "experience_rating": 4,
    "rating_rationale": "User's exact words explaining their rating",
    "experienced_frustration": "yes",
    "frustration_detail": "User's exact words about what was frustrating",
    "best_part": "User's exact words about the best part"
  }
}
```

**Important:** Log all responses as verbatims — the user's actual words, not your summary of them. If the user gives a one-word answer, log the one word. If they give a paragraph, log the paragraph.

### End-of-Day Survey

At the end of the day — or when the user appears to be wrapping up their final session, or when they say something like "okay that's it for today" — trigger the end-of-day survey. This captures the holistic daily experience, not just individual task reactions.

**Present it like this:**

> Before you wrap up — one last set of questions about your overall day with OpenClaw:
>
> 1. **How would you rate your overall experience with OpenClaw today?**
>    *(1 = Poor, 2 = Below average, 3 = Okay, 4 = Good, 5 = Excellent)*
>
> 2. **What's behind that score? What drove your overall impression today?**
>
> 3. **Did you experience anything frustrating today?** *(Yes / No)*
>
> 4. **If yes — what were the frustrating moments? List as many as come to mind.**
>
> 5. **Did anything really impress you or exceed your expectations today?** *(Yes / No)*
>
> 6. **If yes — what stood out? What made it impressive?**
>
> 7. **If you could change one thing about how OpenClaw works, based on today, what would it be?**
>
> 8. **Anything else on your mind about the experience that we haven't covered?**

**Log format for end-of-day surveys:**

```json
{
  "timestamp": "ISO-8601",
  "session_id": "uuid",
  "survey_type": "end_of_day",
  "tasks_completed_today": 7,
  "responses": {
    "overall_rating": 3,
    "rating_rationale": "User's exact words",
    "experienced_frustration": "yes",
    "frustration_details": "User's exact words listing frustrating moments",
    "experienced_delight": "yes",
    "delight_details": "User's exact words about what impressed them",
    "one_change": "User's exact words about what they'd change",
    "additional_thoughts": "User's exact words, or empty if nothing"
  }
}
```

### Survey Delivery Guidelines

- **Be conversational, not clinical.** You're a researcher who respects the participant's time, not a robot administering a form. Brief framing ("Quick check-in on that last task") sets the right tone.
- **If the user declines or brushes off the survey**, log that they declined and move on gracefully. Never push. Note the decline in the observation log — survey non-response is data too.
- **If the user gives very short answers**, that's fine — log them as-is. Don't probe further on post-task surveys. You can gently probe on end-of-day if answers feel incomplete ("Anything specific come to mind on that?").
- **Adapt phrasing slightly to feel natural** in the conversation flow — the questions above are the standard instrument, but you can adjust wording slightly so it doesn't feel robotic if the same survey has been asked many times. The *content* of each question must stay the same — don't change what you're measuring, just smooth the delivery.

## Sub-Agent Architecture

When running in an environment that supports sub-agents (like Cowork or Claude Code), spawn specialized observer agents:

### Observer Agent

Runs passively alongside the main conversation. Its only job is to:
- Watch each interaction turn
- Classify intent, outcome, friction, and sentiment
- Append to `observations.jsonl`
- Flag moments where a survey should fire

Spawn prompt for observer agent:
```
You are a UX research observer. Your job is to watch the interaction that just occurred
and produce a structured observation record. You are not participating in the conversation —
only observing and logging.

Read the latest exchange from the session. Classify it using the observation schema in
~/.uxr-observer/schema/observation.json. Append your record to
~/.uxr-observer/sessions/{today}/observations.jsonl.

CRITICAL: Capture the user's actual words as verbatims. For every notable user statement —
requests, reactions, corrections, praise, complaints — log the exact quote paired with a
short researcher-generated summary header that interprets what the quote represents.

Write a task_context_summary (2-3 sentences) that narrates what happened: what the user
asked for, how OpenClaw handled it, and the outcome. Write this for an audience that wasn't
present — it needs to stand on its own.

Only redact genuinely sensitive content (passwords, API keys, financial details). Everything
else should be captured verbatim.
```

### Survey Agent

Fires after every completed task with the standard 5-question post-task survey. It also fires at end-of-day with the 8-question daily wrap-up. It:
- Writes a task context summary before presenting the post-task survey
- Presents the appropriate survey instrument conversationally
- Logs all responses as verbatims to `surveys.jsonl`
- Notes survey declines as data points

### Distiller Agent (end of day)

Runs at the end of the day (or on-demand when the user asks for their report). Read `references/analysis-framework.md` for the full distillation methodology. In brief:

1. Read all observations and surveys from today
2. For each task, pair the task context summary with its survey responses
3. Organize all user verbatims with researcher-generated summary headers
4. Group verbatims thematically (positive experiences, pain points, expectations, suggestions)
5. Identify patterns, themes, and standout moments across the full day
6. Integrate end-of-day survey responses as a reflective capstone
7. Generate the daily report (see Report Format below)
8. Save to `~/.uxr-observer/reports/`

If sub-agents aren't available (e.g., Claude.ai), perform these roles inline — observe as you go, survey at natural breakpoints, and distill when asked.

## Daily Report Format

Generate reports as Markdown files. The report should be immediately useful — grounded in the user's actual words, not sanitized summaries. Structure:

```markdown
# UXR Daily Report — {DATE}

## Summary
2-3 sentence executive summary of the day's usage patterns and experience quality.

## By the Numbers
- **Tasks completed:** N
- **Post-task surveys completed:** N / N possible (X%)
- **Average post-task satisfaction:** X.X/5
- **Overall day rating:** X/5
- **Tasks with reported frustration:** N
- **Tasks with reported delight:** N

## Task-by-Task Breakdown

For each task observed today, include:

### Task 1: {Brief task description}
**What happened:** {task_context_summary — what the user asked, how OpenClaw responded, what the outcome was}
**Rating:** X/5
**Frustration reported:** Yes/No

**[User's rationale for their rating]**
> "{exact verbatim from rating_rationale}"

**[What frustrated the user]** *(if applicable)*
> "{exact verbatim from frustration_detail}"

**[What the user valued most]**
> "{exact verbatim from best_part}"

**Observed friction signals:** {list from observation}
**Observed sentiment signals:** {list from observation}

---
*(Repeat for each task)*

## Verbatim Gallery

All notable user quotes from the day, organized thematically with researcher-generated headers:

### Positive Experiences
**[Summary header interpreting the quote]**
> "User's exact words"

**[Summary header interpreting the quote]**
> "User's exact words"

### Pain Points & Frustrations
**[Summary header interpreting the quote]**
> "User's exact words"

### Expectations & Mental Models
**[Summary header interpreting the quote]**
> "User's exact words"

### Suggestions & Wishes
**[Summary header interpreting the quote]**
> "User's exact words"

## End-of-Day Reflection

**Overall day rating:** X/5

**[Why the user gave this score]**
> "{verbatim from end-of-day rating_rationale}"

**[Frustrating moments recalled]** *(if reported)*
> "{verbatim from end-of-day frustration_details}"

**[What impressed the user]** *(if reported)*
> "{verbatim from end-of-day delight_details}"

**[What the user would change]**
> "{verbatim from end-of-day one_change}"

**[Additional thoughts]** *(if any)*
> "{verbatim from end-of-day additional_thoughts}"

## Patterns & Insights

### What's Working Well
- Insight (grounded in specific tasks and verbatims from today)

### Recurring Pain Points
- Pain point (with frequency count and supporting verbatims)

### Emerging Themes
- Any patterns across tasks that suggest deeper UX issues or opportunities

## Recommendations
Based on today's data:
1. Recommendation (tied to specific evidence)
2. Recommendation

---
*This report was generated locally by UXR Observer. No data has been transmitted externally.*
*Report file: ~/.uxr-observer/reports/{filename}*
*To share: ask OpenClaw to email it, or download and share it yourself.*
```

## Sharing Reports

When the user wants to share a report:

1. **If the user asks you to email it** — use whatever email or messaging tools are available to send it to whomever they specify. This is user-initiated sharing and is perfectly fine. Always confirm the recipient before sending.
2. **If the user wants to download it** — copy the report file to `/mnt/user-data/outputs/` so they can access it.
3. **Never send reports proactively.** Don't email, upload, or transmit any data unless the user explicitly asks you to in that moment. "Send me my daily report every evening" is fine as a standing instruction. Sending it somewhere the user never asked for is not.

The principle is simple: **every transmission requires user intent.** The user is always in control of where their data goes.

## First Run Setup

On first activation, do the following:

1. Create the `~/.uxr-observer/` directory structure
2. Generate a random `participant_id` and save `config.json`
3. Briefly explain to the user what this skill does:

> "Hey — the UXR Observer skill is now active. Here's what it does: I'll be passively observing how our interactions go — what you ask for, how well it works, any friction points — and capturing your words along the way. After every task, I'll ask you 5 quick questions about the experience (takes about 30 seconds). At the end of the day, there's a slightly longer wrap-up survey. Then I'll compile everything into a daily report with your verbatim feedback, insights, and patterns. All data stays local unless you ask me to send it somewhere. You can pause or stop the study anytime."

4. Start observing.

## Commands the User Can Use

Respond to these natural language commands:

- **"Show me today's observations"** → Display the current day's observation log
- **"Generate my daily report"** / **"Give me my report"** → Run the distiller immediately
- **"Email my report to [person/address]"** → Generate the report and send it via email to the specified recipient
- **"Send me my report"** → Generate and email the report to the user
- **"Run the end-of-day survey"** → Trigger the end-of-day wrap-up survey immediately
- **"Pause the study"** / **"Stop observing"** → Set `study_active: false`, stop logging
- **"Resume the study"** → Set `study_active: true`, resume
- **"What are you tracking?"** → Full transparency — explain everything and offer to show raw data
- **"Show me the raw data"** → Display the JSONL logs directly
- **"Delete my data"** → Delete all files in `~/.uxr-observer/` after confirmation
- **"Show me trends"** → If multiple days of data exist, generate a cross-day trend analysis
- **"Skip the survey"** → Acknowledge, log the decline, move on without pushing