Xiaohongshu Search Full

SKILL.md

---
name: xiaohongshu-search-full
description: "Search Xiaohongshu (XHS / RedNote) notes by keyword with full field extraction including body text, topics/tags, image list URLs, video stream URL, publish timestamp, and all engagement stats (likes, collects, comments, shares). Supports all page filter options: sort order (general, latest, most liked, most commented, most collected), note type (image-text, video), publish time range (within 1 day, 1 week, 6 months), search scope (seen, unseen, followed), and location distance (same city, nearby). Use when user mentions search xiaohongshu notes, xhs keyword search, rednote note search, search xiaohongshu posts, scrape xhs search results, xiaohongshu note discovery, rednote content search, collect xhs notes by keyword, xiaohongshu topic search, xhs note body text, xiaohongshu video notes, xiaohongshu image notes, rednote post filter, xhs search with filters, xiaohongshu full note data, xhs note details from search, extract rednote posts, xiaohongshu content monitoring, xhs search scrape."
---

# Xiaohongshu — Search Notes (Full Fields)

> keyword + filters → note list with full metadata (title, body, images, video URL, tags, stats) + detail enrichment

## Language

All process output to user (progress updates, process notifications) follows the user's language.

## Objective

Search Xiaohongshu notes by keyword, apply page filter options, and extract the maximum available fields from both the search results list and individual note detail pages.

## Prerequisites

- Browser opened to `https://www.xiaohongshu.com/search_result/?keyword={keyword}`
- User is logged in (avatar or username visible in the left sidebar)

## Phase 0: Collect User Inputs

**First action**: check whether the search keyword is already present in the user's message.

**Keyword is present** (e.g., "搜索 BrowserAct", "search for AI tools", "/xiaohongshu-search-full blockchain") → use it directly, skip to Pre-execution Checks.

**Keyword is NOT present** → STOP. Do NOT output "ready" or proceed with any execution. Ask the user for the keyword:
- If the **AskUserQuestion** tool is available → call it immediately with:
  - Question 1 (required): "请问您想在小红书上搜索什么关键词？" 
  - Question 2 (optional): pages needed (default: first page only)
  - Question 3 (optional): filters — sort order, note type, publish time range
- If AskUserQuestion is NOT available → output the following text and wait for the user's reply before doing anything else:
  > "请问您想搜索什么关键词？（如需指定页数或筛选条件，也可一并告知）"

Do NOT guess, infer, or assume any keyword. Wait for the user's explicit reply.

## Pre-execution Checks

### 1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke `browser-act` via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

### 2. Login Verification

If login status for Xiaohongshu has been confirmed in the current session → skip this step.

Otherwise: open `https://www.xiaohongshu.com` and observe the left sidebar:
- User avatar or "Me" entry visible → logged in, continue execution
- "Login" button visible → not logged in, inform the user that login is required, use `remote-assist` to let the user scan the QR code

User refuses or cannot log in → terminate execution.

## Capability Components

> This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page, never bypassing authentication or access controls. Its role is equivalent to copy-pasting on the user's behalf — the data is already on screen, automation merely saves time. JS code is encapsulated in Python files under the `scripts/` directory, invoked via `python scripts/xxx.py {params} | browser-act --session <name> eval --stdin`.

Below are all atomic capabilities discovered and verified during the exploration phase, listed by command template with parameters. Simply invoke them as needed — no need to read `scripts/*.py` source code or re-verify. Only inspect scripts when execution fails for troubleshooting. Combine freely as needed during execution.

### DOM: extract search feeds list (preferred method)

Navigate to the search page, then read the rendered note list directly from Vue state. This is the **preferred method** because it reflects exactly what the page renders — including exact-match notes that the search API omits due to semantic expansion.

1. `navigate https://www.xiaohongshu.com/search_result/?keyword={keyword}`
2. `wait stable`
3. (optional) Apply filters — see **AI Workflow: apply filters** below
4. `python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin` — retrieves all page results. **Do NOT pass `--keyword` for a full keyword search** (title + body); see **AI Workflow: keyword search (title + body match)** below. Pass `--keyword {keyword}` only when title-only pre-filtering is explicitly required.

Parameters:
- `--keyword {keyword}` (optional): plain text keyword — when passed, `items` contains only title-matched notes and `keyword_count` is populated. Omit to get all rendered notes (required for body-text matching).

Output example:
```json
{
  "total_count": 40,
  "keyword_count": 3,
  "has_more": true,
  "items": [
    {
      "id": "6a422f88000000001603cdd3",
      "xsec_token": "ABbCLyAhrP9qgixS_rCW...",
      "note_url": "https://www.xiaohongshu.com/explore/6a422f88000000001603cdd3?xsec_token=ABbCLy...%3D&xsec_source=pc_search",
      "type": "video",                           // "normal" = image-text, "video" = video note
      "title": "note title text here",
      "publish_date": "06-29",                   // human-readable date string (not exact timestamp)
      "cover_url": "http://sns-webpic-qc.xhscdn.com/...",
      "liked_count": "3",
      "collected_count": "7",
      "comment_count": "0",
      "shared_count": "0",
      "author_nickname": "author nickname here",
      "author_id": "670caaaa000000001d0239d1",
      "author_avatar": "https://sns-avatar-qc.xhscdn.com/..."
    }
  ]
}
```

- `total_count`: all notes rendered by the page (includes semantically expanded results)
- `keyword_count`: notes whose **title** fuzzy-matches the keyword (Levenshtein edit distance ≤ 20% of keyword length, min 1 edit)
- `items`: the fuzzy-filtered list (only notes whose title fuzzy-matches the keyword)
- `keyword_count: null` means no `--keyword` was passed and `items` = all notes

Fuzzy matching details: `--keyword` uses Levenshtein edit distance with threshold `max(1, floor(kw.length * 0.2))` per sliding window over each word in the title. This catches typos, capitalization variants (e.g., `linfox` → `LinkFox`, `linxfox`), and minor spelling differences within the threshold.

Note: `has_more` indicates whether more pages exist. `body text (desc)`, `topics (tagList)`, `video stream URL`, and `exact publish timestamp (ms)` are NOT in this response — use the detail component below to obtain them. When `--keyword` is passed, matching is title-only (fuzzy); for title + body matching, omit `--keyword` and follow **AI Workflow: keyword search (title + body match)** below.

Error handling: if `error: true`, verify `wait stable` completed and the page is not a login gate. Reload and retry once.

### AI Workflow: keyword search (title + body match)

Run this workflow whenever a keyword is provided. It finds all notes where the keyword appears in either the **title** OR the **body text (desc)**.

**Step 1 — Extract all page results** (no `--keyword` flag):
```bash
python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin
```
→ `all_items` list (`total_count` items, no pre-filtering)

**Step 2 — Title filter** (in-memory, no extra requests):
- `title_matches` = items where `keyword` fuzzy-matches `title` (use same Levenshtein threshold as `--keyword`: `max(1, floor(len(keyword) * 0.2))` edits per word, case-insensitive, exact substring first)
- `candidates` = remaining items (ALL items not matched by title fuzzy match)
- Inform user: "Found {len(title_matches)} title match(es). Checking body text of {len(candidates)} candidates..."
- **CRITICAL: When `title_matches == 0`, do NOT stop. Zero title matches means body-text checking is MORE important, not less — the keyword may appear only in the body. Proceed to Step 3 unconditionally unless `total_count == 0`.**

**Step 3 — Body-text check** (detail page per candidate, skip only if `candidates` is empty):
For each item in `candidates`:
1. `browser-act --session <name> navigate https://www.xiaohongshu.com/explore/{id}?xsec_token={xsec_token}&xsec_source=pc_search`
2. `browser-act --session <name> wait stable`
3. `python scripts/extract-note-detail.py {id} | browser-act --session <name> eval --stdin`
4. If `keyword` fuzzy-matches (case-insensitive, same Levenshtein threshold) in `desc` → add to `body_matches`
5. Wait 2–3 seconds before next request.

**Step 4 — Report**:
- `final_results` = `title_matches` + `body_matches` (in original page order, deduplicated by `id`)
- Inform user: "{len(title_matches)} title match(es) + {len(body_matches)} body-only match(es) = {total} notes containing '{keyword}' (from {total_count} page results)"
- Clearly label each result: `match_type: "title"` or `match_type: "body"`

**Edge cases**:
- `title_matches == 0 AND body_matches == 0` → inform user: no notes found containing `{keyword}` in title or body text; suggest checking login status or trying a different keyword
- `total_count == 0` → inform user: no results at all; suggest checking login or trying a different keyword

### Network Capture: search notes list (fallback method)

Use as fallback only when the DOM feeds extraction above fails. Navigate to the search page and read results from the `so.xiaohongshu.com` API response captured in browser traffic.

**Important limitation**: the search API applies semantic expansion — for niche or brand-specific keywords, exact-match notes may be absent or pushed to later pages in the API response even when they appear prominently in the rendered page. Always prefer the DOM feeds method above.

1. `navigate https://www.xiaohongshu.com/search_result/?keyword={keyword}`
2. `wait stable`
3. (optional) Apply filters — see **AI Workflow: apply filters** below
4. `network requests --type xhr,fetch --filter so.xiaohongshu`
5. Identify the request with URL containing `/api/sns/web/v2/search/notes`
6. `network request <id>`
7. Parse `response_body` JSON — field paths: `data.items[n].id`, `data.items[n].xsec_token`, `data.items[n].note_card.display_title`, `data.items[n].note_card.interact_info.*`

Note: `data.has_more` indicates whether more pages exist.

Error handling: If no request matching `/api/sns/web/v2/search/notes` is found, check that `wait stable` completed and the page is a search result page (not a login gate). Reload the page and retry once.

### DOM: extract note detail (body, topics, video URL, exact timestamp)

Navigate to the note detail page and extract enriched fields from the Vue SSR state:

1. `navigate https://www.xiaohongshu.com/explore/{note_id}?xsec_token={xsec_token}&xsec_source=pc_search`
2. `wait stable`
3. `python scripts/extract-note-detail.py {note_id} | browser-act --session <name> eval --stdin`

Parameters:
- `{note_id}`: note ID from the search list result (`id` field)
- `{xsec_token}`: security token from the search list result (`xsec_token` field)

Output example:
```json
{
  "noteId": "694692cc000000001d039ea3",
  "title": "note title here",
  "desc": "note body text here",                // full body text
  "type": "video",                               // "normal" or "video"
  "time": 1766232780000,                         // exact publish timestamp (milliseconds)
  "ipLocation": "Fujian",                        // IP location, null if unavailable
  "userId": "59e9658411be10340721cd79",
  "nickname": "author nickname here",
  "avatar": "https://sns-avatar-qc.xhscdn.com/avatar/...",
  "likedCount": "0",
  "collectedCount": "0",
  "commentCount": "0",
  "shareCount": "0",
  "tagList": [
    {"name": "topic name", "type": "topic", "id": "123"}   // topics/hashtags
  ],
  "imageList": [
    {"url": "http://sns-webpic-qc.xhscdn.com/...", "width": 720, "height": 960}
  ],
  "videoUrl": "http://sns-video-v6.xhscdn.com/stream/1/110/259/..."   // null for image-text notes
}
```

Error handling: if `error: true` is returned, verify the page URL is a note detail page, `wait stable` has completed, and `note_id` matches the URL. If `noteDetailMap` is empty after wait, try `wait --selector ".note-content" --state attached --timeout 15000` then re-run the extraction.

### AI Workflow: apply filters (before search capture)

Run this workflow **before** step 4 of the search capture component when filters are needed. Open the filter panel and select desired options, then wait for the filtered search request to fire.

1. `state` — locate the "Filter" button (labeled with filter icon) in the top-right area of the search content area → `click <index>`
2. Wait for filter panel to appear (visible on the right side of the page)
3. For **sort order** — `state` locate the desired sort tag in the "Sort By" row → `click <index>`
4. For **note type** — `state` locate the desired type tag in the "Note Type" row → `click <index>`
5. For **publish time** — `state` locate the desired range tag in the "Publish Time" row → `click <index>`
6. For **search scope** — `state` locate the desired scope tag in the "Search Scope" row → `click <index>`
7. For **location distance** — `state` locate the desired distance tag in the "Location Distance" row → `click <index>`
8. `wait stable`
9. Proceed with `network requests --type xhr,fetch --filter so.xiaohongshu` to capture the filtered results

Filters are sent in the POST body `filters` array. Each filter entry has shape `{"type": "{filter_id}", "tags": ["{selected_value}"]}`. See Enum Parameters for all verified values.

### Composite: search + enrich (search list → detail page loop)

Use when full fields including body text, topics, video URL, and exact timestamp are needed for **all** notes regardless of keyword filtering. Also used as a sub-step in **AI Workflow: keyword search (title + body match)** for the body-text candidate checking phase.

1. Navigate to search page and extract feeds via DOM (preferred): `python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin`
2. Parse `items` array from result to get note list with `id` and `xsec_token`
3. For each note:
   a. `navigate https://www.xiaohongshu.com/explore/{id}?xsec_token={xsec_token}&xsec_source=pc_search`
   b. `wait stable`
   c. `python scripts/extract-note-detail.py {id} | browser-act --session <name> eval --stdin`
   d. Merge detail fields with feeds list fields (cover from feeds is higher resolution than detail page cover)
4. Save merged result per note

Recommended batch delay: 2–3 seconds between detail page requests to avoid anti-scraping triggers.

## Enum Parameters

Note: Some API parameter values below are platform-internal Chinese strings — they must be sent to the server exactly as shown (the server rejects English substitutes). All surrounding documentation text is English per Skill standards.

[AI] sort_type — `filters` array entry `{"type": "sort_type", "tags": ["{value}"]}`. Verified values from filter API response:

| UI Label | API Value (send as-is in `tags`) |
|---|---|
| General (default) | `general` |
| Latest | `time_descending` |
| Most Liked | `popularity_descending` |
| Most Commented | `comment_descending` |
| Most Collected | `collect_descending` |

[AI] filter_note_type — `filters` array entry `{"type": "filter_note_type", "tags": ["{value}"]}`. API values are platform-internal Chinese strings (non-substitutable):

| UI Label | API Value |
|---|---|
| All (default) | `不限` |
| Video | `视频笔记` |
| Image-text | `普通笔记` |

[AI] filter_note_time — `filters` array entry `{"type": "filter_note_time", "tags": ["{value}"]}`. API values are platform-internal Chinese strings:

| UI Label | API Value |
|---|---|
| All (default) | `不限` |
| Within 1 day | `一天内` |
| Within 1 week | `一周内` |
| Within 6 months | `半年内` |

[AI] filter_note_range — `filters` array entry `{"type": "filter_note_range", "tags": ["{value}"]}`. API values are platform-internal Chinese strings:

| UI Label | API Value |
|---|---|
| All (default) | `不限` |
| Seen | `已看过` |
| Unseen | `未看过` |
| Followed | `已关注` |

[AI] filter_pos_distance — `filters` array entry `{"type": "filter_pos_distance", "tags": ["{value}"]}`. API values are platform-internal Chinese strings:

| UI Label | API Value |
|---|---|
| All (default) | `不限` |
| Same city | `同城` |
| Nearby | `附近` |

Acquisition method for all enums: open filter panel via `state` + `click` on the Filter button, read filter option rows; or read directly from the `edith.xiaohongshu.com/api/sns/web/v1/search/filter?keyword={keyword}` GET response captured on initial page load.

## Pagination

**DOM Pagination** (search feeds): `scroll down --amount 3000` → `wait stable` → re-run `python scripts/extract-search-feeds.py | browser-act --session <name> eval --stdin` (result count accumulates as page loads more). Each scroll loads ~20 more results. Termination: `has_more` is `false` in extraction output.

**Network Capture Pagination** (fallback only): `scroll down --amount 3000` → `wait stable` → re-read `network requests --filter so.xiaohongshu` → `network request <id>` (new request will appear with `page` incremented). Termination: `data.has_more` is `false` in the latest response.

## Success Criteria

DOM feeds extraction: `result.count >= 1 AND result.items[0].id is non-null AND result.items[0].title is non-null`

For detail extraction: `result.noteId is non-null AND result.title is non-null`

## Known Limitations

- Search and detail extraction require login; without login the page shows a QR code overlay and returns no data
- `xsec_token` must correspond to the `note_id`; tokens from other sources result in redirect or empty state
- **Search API semantic expansion**: the `/v2/search/notes` API applies semantic query expansion — for niche or brand-specific keywords, exact-match notes may be absent from or pushed to later pages in the API response even when they appear prominently in the rendered page. The DOM feeds extraction (`extract-search-feeds.py`) reads `window.__INITIAL_STATE__.search.feeds` which mirrors exactly what the page renders, and is not affected by this limitation
- `publish_date` in feeds list gives a human-readable date string (e.g., `"2025-10-29"` or `"06-27"`) but not a precise timestamp — go to the detail page for the exact millisecond timestamp
- `filter_pos_distance` values for same-city and nearby require the browser's location permission to be enabled; without permission these filters may return empty results
- Body text (`desc`) and topics (`tagList`) are only available from the detail page SSR state, not from the search feeds list
- Video stream URLs contain time-limited signed tokens (`sign=...&t=...`) — links expire; re-navigate to generate fresh URLs

## Execution Efficiency

- **Batch orchestration**: Write a bash script to loop through the command templates serially within a single session; do not parallelize within one browser (prone to triggering anti-scraping restrictions). Refer to rate information in "Known Limitations" above to add appropriate intervals. To increase throughput, open multiple stealth browser sessions and distribute work across them — each session has an independent fingerprint so rate limits apply per session
- **Test before batch execution**: After writing a batch script, you must first test with 1-2 items to verify the script runs correctly; only then run the full batch. Never skip testing and execute in batch directly
- **Reduce redundant pre-operations**: When multiple steps depend on the same prerequisite state, complete them in batch under that state to avoid repeatedly establishing the same state
- **Error resumption**: Save results item by item during batch processing; on failure, resume from the breakpoint rather than starting over

## Experience Notes

Path: `{working-directory}/browser-act-skill-forge-memories/xiaohongshu-search-full-xiaohongshu-search-full.memory.md` (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

**Before execution**: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

**After execution**: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:
`{YYYY-MM-DD}: {what happened} → {conclusion}`

Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.