Site Watch

SKILL.md

---
slug: site-watch
name: Site Watch
display_name: Site Watch / 网页变更监控
description: Lightweight CLI-based web page change monitor with AI summaries and multi-channel notifications
author: Golden Bean (coder)
version: 1.0.0
license: MIT
type: skill
tags:
  - web-monitor
  - change-detection
  - price-tracker
  - notification
model: deepseek
created_at: 2026-06-14
---

# Site Watch (site-watch)

> **Version**: 1.0.0 | **Category**: Utilities / Automation
> **Slug**: `site-watch` | **Runtime**: Node.js 18+
> **Dependencies**: cheerio, better-sqlite3, node-cron, node-notifier

## Description

A lightweight CLI-based web page change monitor that periodically fetches pages, detects content changes,
generates intelligent AI summaries (powered by DeepSeek), and pushes notifications through multiple channels.
All data stays local — zero cloud dependency.

**Core workflow**: Add a URL → Scheduler fetches periodically → Detects changes → AI summarizes →
Multi-channel notification → You act on it.

---

## First-Success Path

**Goal**: First monitoring target added and one change detected within 60 seconds.

```
Step 1: clawhub install site-watch
Step 2: site-watch add --url "https://jsonplaceholder.typicode.com/posts/1" --name "Test Page"
Step 3: Internal pipeline:
  a. index.js CLI parses --url and --name arguments
  b. target-manager.js creates target entry in SQLite DB
  c. fetcher.js HTTP GET with automatic retry
  d. content-extractor.js cheerio HTML → text extraction
  e. change-detector.js SHA-256 hash baseline snapshot
  f. AI summary generated (if DEEPSEEK_API_KEY set)
  g. notifier.js prints confirmation with initial snapshot
Step 4: User sees "✅ Monitor Added" with preview
Step 5: Value achieved — first target is being watched
```

**Key Metrics**: Install→First target < 10s, Change detection < 5s, Startup < 1s.

---

## Installation

```bash
clawhub install site-watch
```

Dependencies are auto-installed. For headless browser support (JS-rendered pages):

```bash
cd ~/.openclaw/skills/site-watch && npm install playwright-core
```

---

## Usage

All commands follow the pattern:

```
site-watch <action> [options]
```

### Add a monitoring target

```bash
site-watch add --url "https://example.com/page" [--name "My Page"] [--selector ".content"] [--frequency 1h] [--tags tag1,tag2]
```

### List all targets

```bash
site-watch list [--status active|paused|error|all]
```

### Start/stop the scheduler

```bash
site-watch start
site-watch stop
```

### Manual check

```bash
site-watch check --name "My Target"
```

### View history

```bash
site-watch history --name "My Target" [--since 2026-01-01]
```

### Export data

```bash
site-watch export --name "My Target" --format csv
```

### Full status overview

```bash
site-watch status
```

---

## Actions Reference

| Action     | Description                                              |
|------------|----------------------------------------------------------|
| `add`      | Add a new URL to monitor, fetches initial snapshot       |
| `remove`   | Remove a monitored target and its history                |
| `list`     | List all monitored targets with status                   |
| `status`   | Scheduler health + target summary                        |
| `start`    | Start the background scheduler process                   |
| `stop`     | Gracefully stop the scheduler                            |
| `check`    | Manually check a single target for changes               |
| `history`  | View change history for a target                         |
| `export`   | Export change history as JSON/CSV/Markdown               |

---

## Options

| Option            | Type    | Default    | Description                                 |
|-------------------|---------|------------|---------------------------------------------|
| `--url`           | string  | (required) | Target page URL                             |
| `--name`          | string  | page title | Display name for the target                 |
| `--selector`      | string  | full page  | CSS selector for area monitoring            |
| `--frequency`     | string  | `1h`       | Check frequency (`1m`,`5m`,`15m`,`30m`,`1h`,`6h`,`12h`,`24h`) |
| `--sensitivity`   | string  | `normal`   | Change sensitivity (`low`,`normal`,`high`)  |
| `--preset`        | string  | `auto`     | Platform preset (`jd`,`taobao`,`pdd`,`dewu`,`bilibili`,`zhihu`,`xiaohongshu`,`github`,`auto`,`none`) |
| `--tags`          | list    | —          | Comma-separated tags for categorization     |
| `--render-js`     | flag    | off        | Use headless browser for JS-rendered pages  |
| `--no-summary`    | flag    | off        | Disable AI-generated change summaries       |
| `--timeout`       | number  | `30000`    | Request timeout in milliseconds             |
| `--format`        | string  | `json`     | Export format (`json`,`csv`,`markdown`)     |
| `--data-dir`      | path    | default    | Custom data storage directory               |

---

## Sample Prompts

### 1. Price monitoring for JD.com product

```
site-watch add \
  --url "https://item.jd.com/100012345.html" \
  --name "京东 iPhone 15 Pro Price" \
  --preset jd \
  --frequency 1h \
  --tags shopping,apple
```

Expected output:
```
✅ Monitor Added

  📋 Target: 京东 iPhone 15 Pro Price
  🔗 URL: https://item.jd.com/100012345.html
  🏷️  Tags: shopping, apple
  🎯 Selector: .summary-price .price
  🌐 Platform: jd
  ⏱️  Frequency: every 60 minute(s)

  📸 Initial Snapshot Preview:
  "iPhone 15 Pro 256GB 暗紫色 ¥7,999 现货"

  💡 Use `site-watch start` to begin periodic monitoring
  💡 Use `site-watch check --name "京东 iPhone 15 Pro Price"` to check for changes
```

### 2. Content tracking for a blog/article page

```
site-watch add \
  --url "https://example.com/blog/latest-post" \
  --name "Tech Blog Updates" \
  --selector "article.main-content" \
  --frequency 6h \
  --tags blog,tracking
```

### 3. Job listing monitoring on BOSS直聘

```
site-watch add \
  --url "https://www.zhipin.com/web/geek/job?query=前端工程师" \
  --name "Frontend Jobs" \
  --frequency 12h \
  --tags job,frontend
```

### 4. Check status of all monitored targets

```
site-watch list
```

### 5. Start the background scheduler

```
site-watch start
```

---

## Data Storage

All data is stored locally under `~/.openclaw/data/site-watch/`:

| File/Directory   | Purpose                                      |
|------------------|----------------------------------------------|
| `config.json`    | Global configuration (notifications, etc.)   |
| `site-watch.db`  | SQLite database (targets, snapshots, changes)|
| `targets/`       | Per-target config (cookies/headers encrypted)|
| `.encryption-key`| AES-256-GCM key for credential encryption    |
| `scheduler.pid`  | PID file for scheduler process tracking      |

**Security**: Sensitive fields (cookies, custom headers, webhook URLs) are encrypted using AES-256-GCM
before being written to disk. Config files are created with `0600` permissions.

---

## AI Summary

When a change is detected, an optional AI summary can be generated to explain what changed in natural language.
The tool supports:

1. **DeepSeek** (preferred) — set `DEEPSEEK_API_KEY` environment variable
2. **Custom LLM endpoint** — set `SITEWATCH_LLM_ENDPOINT`, `SITEWATCH_LLM_API_KEY`, `SITEWATCH_LLM_MODEL`
3. **Template-based fallback** — built-in, no API needed

Only the change diff (max 3000 chars) is sent to the LLM — never the full page content. PII (phone numbers,
emails, ID numbers) is automatically masked before sending.

---

## Platform Presets

| Preset     | Platform       | Key Selectors                |
|------------|----------------|------------------------------|
| `jd`       | JD.com         | `.summary-price .price`      |
| `taobao`   | Taobao/Tmall   | `.tm-price`                  |
| `pdd`      | Pinduoduo      | `.goods-price`               |
| `dewu`     | Dewu/Poizon    | `.price-text`                |
| `bilibili` | Bilibili       | `.video-info-desc`           |
| `zhihu`    | Zhihu          | `.Post-RichText`             |
| `xiaohongshu` | Xiaohongshu | `.note-content`              |
| `github`   | GitHub         | `.release-body`              |

Use `--preset auto` (default) to auto-detect the platform from the URL, or `--preset none` for manual
selector specification.

---

## Architecture

```
index.js (CLI entry)
  ├── target-manager.js    — CRUD for monitored targets
  ├── fetcher.js           — HTTP requests with retry & anti-bot measures
  ├── content-extractor.js — HTML → text via cheerio + CSS selectors
  ├── noise-filter.js      — Timestamp/ads/counter removal
  ├── change-detector.js   — SHA-256 hash + text diff + sensitivity
  ├── ai-summarizer.js     — LLM-powered change summary (DeepSeek)
  ├── scheduler.js         — Cron-based periodic checker
  ├── notifier.js          — Multi-channel dispatch (terminal, system, webhook)
  ├── history-store.js     — SQLite persistence with zlib compression
  ├── config.js            — Config manager + AES-256-GCM encryption
  └── security.js          — SSRF protection, PII masking, robots.txt
```

---

## Error Codes

| Code   | Meaning                    | Recovery                             |
|--------|----------------------------|--------------------------------------|
| `E001` | Invalid URL format         | Provide a valid URL                  |
| `E002` | DNS/network error          | Check connectivity / DNS             |
| `E003` | Request timeout            | Increase --timeout                   |
| `E004` | HTTP error (4xx/5xx)       | Check URL accessibility              |
| `E005` | CSS selector no match      | Verify selector / use full page      |
| `E008` | Rate limited (429)         | Reduce frequency                     |
| `E010` | AI summary failed          | Falls back to text diff              |
| `E013` | Scheduler conflict         | Stop first, then start               |
| `E014` | Target not found/duplicate | Check target name/id                 |

---

## License

MIT — Part of the Golden Bean skill collection.