Web Watcher Pro

SKILL.md

---
name: web-watcher-pro
description: "Web Change Monitor — Generic webpage monitoring tool. Configure URL list → Skill checks for changes at set frequency → Feishu push notifications. Not tied to any platform, fully generic. Triggers: webpage monitor, page change detection, URL monitor, price change monitor, competitor monitoring, website update alert, inventory monitoring, stock change detection, website monitor."
override-tools: []
---

# Web Watcher Pro

Configure any URL → Skill checks for changes at set frequency → Feishu notification.

Fully generic tool, not tied to any platform. Use cases: competitor new product alerts, price monitoring, inventory tracking, content change detection, forum thread monitoring.

## Quick Start

### Add a Monitored URL

```
User: Monitor this page: https://example.com/product/12345
```

Skill:
1. Fetches page, computes content hash
2. Asks for detection mode and frequency (or uses defaults)
3. Saves monitoring task, begins checking

### Check Status

```
User: Show my monitored URLs
User: Which URLs have changed?
```

### Remove Monitor

```
User: Remove monitoring for https://example.com/product/12345
```

---

## Detection Modes

| Mode | Description | Use Case |
|------|-------------|----------|
| `hash` | MD5 hash of full HTML, triggers on any change | General, any page |
| `keyword` | Triggers when keyword appears/disappears | Inventory, price, specific content |
| `selector` | CSS selector extracts specific DOM elements for comparison | List pages (product listings, search results) |
| `regex` | Regex-defined trigger condition | Complex pattern matching |

### Examples

```
User: Monitor this page, alert me when price drops below 99
[URL]

User: Use keyword mode, alert when product name contains "New Arrival"
[URL]
```

---

## Tiered Features

| Feature | FREE | PRO |
|---------|:----:|:---:|
| Monitored URLs | 3 | Unlimited |
| Check frequency | Every 24h | Every 1h |
| Detection mode | Hash only | Hash + Keyword + Selector + Regex |
| Change history | — | 30 days |
| Feishu push | — | Yes |
| Price | Free | $0.01/call |

---

## Detection Modes Detail

### Hash Mode
MD5 hash of full page HTML. Triggers on any content change.

### Keyword Mode
Monitors for keyword appearance/disappearance. Case-insensitive.

### Selector Mode
CSS selector extracts specific DOM elements. Compares extracted text between checks.

### Regex Mode
Regex pattern matched against HTML. Triggers on pattern match change.

---

## Change History

```
User: What pages have changed recently?
User: Show change history for https://xxx.com
```

Returns: change timestamp, change summary, time since last change.

---

## Core Script

See `scripts/monitor.py` for full implementation:

```python
from scripts.monitor import WebMonitor

monitor = WebMonitor(tier="pro")

monitor.add_task(
    url="https://example.com/product/123",
    name="Product A Monitor",
    mode="hash",
    frequency="6h",
)

monitor.check_all()  # Triggers Feishu push on changes
monitor.list_tasks()
monitor.remove_task(url="https://example.com/product/123")
```

---

## Technical Implementation

- **Fetching**: Playwright (headless) with random UA and anti-detection delays
- **Detection**: MD5 hash / keyword match / CSS selector / regex
- **Storage**: SQLite at `/tmp/web-watcher-pro/history.db`
- **Push**: Feishu IM notifications with customizable templates
- **Anti-ban**: Request intervals + random delays + 3x auto-retry

---

## Security Notes

- **SSRF Protection**: `fetch_page()` validates all URLs before sending to Playwright. Blocks: non-HTTP(S) schemes (file://, ftp://, data:, javascript:, etc.), localhost, 127.0.0.1, private IP ranges (10.x.x.x, 172.16-31.x.x, 192.168.x.x), link-local (169.254.x.x including AWS metadata 169.254.169.254), and IPv6 localhost. Unsafe URLs return `None` instead of triggering a network request.
- **Subprocess execution**: Uses `node -e` subprocess for Playwright browser automation (anti-detection scraping). Node.js required. Timeout: 30s. Subprocess uses list form (not shell=True), eliminating command injection risk.
- **Data storage**: Uses `/tmp/web-watcher-pro/` for SQLite DB and config (no home directory write).
- **Billing data**: `FEISHU_USER_ID` transmitted to `skillpay.me/api/v1/billing` for per-call charging.

---

## Billing

- Billing via `skillpay.me/api/v1/billing/charge`
- User data transmitted to SkillPay for billing identification
- $0.01 USD per check call (PRO tier)

---

## Required Environment Variables

| Variable | Description |
|----------|-------------|
| `FEISHU_USER_ID` | User open_id for billing |
| `SKILL_BILLING_API_KEY` | SkillPay Builder API Key |
| `SKILL_BILLING_SKILL_ID` | SkillPay Skill ID (default: web-watcher-pro) |

---

## Common Errors

| Error | Cause | Solution |
|-------|-------|----------|
| `Failed to fetch page` | Page blocked or unavailable | Check URL accessibility |
| `Invalid mode` | Unsupported detection mode | Use: hash, keyword, selector, regex |
| `TASK_LIMIT_EXCEEDED` | URL count exceeds tier limit | Upgrade or remove existing URLs |