back
loading skill details...
Collects every tweet in an X (Twitter) conversation thread given a conversation id (root tweet id) — the focal tweet plus all replies, sub-replies, and quote chains — and returns normalized per-tweet data with text, author, engagement counts, media, hashtags, mentions, in_reply_to mapping, and curso
---
name: x-tweet-by-conversation
description: "Collects every tweet in an X (Twitter) conversation thread given a conversation id (root tweet id) — the focal tweet plus all replies, sub-replies, and quote chains — and returns normalized per-tweet data with text, author, engagement counts, media, hashtags, mentions, in_reply_to mapping, and cursor for pagination. Use when user mentions Twitter conversation, X conversation thread, conversation_id, thread scraper, scrape Twitter replies, all replies to a tweet, replies under a tweet, sub-replies, nested replies, thread harvester, get replies of a tweet, scrape comments on Twitter, scrape comments on X, full thread extraction, conversation export, conversation tree, reply chain, thread dump, X tweet thread, twitter thread scrape, comment scraping twitter, comment scraping x, focal tweet plus context, root tweet plus replies. Also applies to sentiment analysis on a single viral tweet, controversy mapping, harvesting community Q&A threads, capturing AMA threads, recovering long-running discussions, and any paginated bulk reply collection driven by a conversation id."
---
# X — Tweets by Conversation
> Conversation id (root tweet id) → normalized list of every tweet in the thread (focal tweet + replies + sub-replies), with author, engagement, media, cursor.
## Language
All process output to user (progress updates, process notifications) follows the user's language.
## Objective
Given an X conversation id (which equals the root tweet id), collect the focal tweet and every reply / sub-reply visible to the logged-in session, returning structured per-tweet data with pagination cursors.
## Prerequisites
- Active X session in the browser (left sidebar shows logged-in avatar / @handle).
- Network capture is enabled in the browser-act session.
## Pre-execution Checks
### 1. Tool Readiness
If browser-act has been confirmed available in the current session → skip this step.
Invoke `browser-act` via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.
### 2. Login Verification
If login status for X has been confirmed in the current session → skip this step.
Otherwise: open `https://x.com` and observe the left sidebar:
- User avatar or @handle visible → logged in, continue
- "Sign in" / "Log in" prompt visible → not logged in, inform the user and assist the login flow
User refuses or cannot log in → terminate execution.
## Capability Components
> This Skill's operational boundary = what the user can manually do in their browser. It only reads tweet data already shown to the user, never bypassing authentication. The browser's own JS signs the GraphQL request; the Skill triggers it via URL navigation and reads the response from network traffic. Python scripts under `scripts/` only build URLs and parse responses — they do not call X directly. Run them through the bash tool.
### Network Capture: full conversation thread
Step 1 — build the tweet-detail URL for the conversation root:
`URL=$(python scripts/build-conversation-url.py '{conversation_id}' [--handle {handle}])`
Parameters:
- `conversation_id` (positional): the X conversation id, which equals the root tweet id (the same number you would see in `https://x.com/i/status/<conversation_id>`).
- `--handle`: the root tweet's author handle if known; if unknown, leave the default — `x.com/i/status/<id>` resolves to the same focal tweet because X canonicalises the URL.
Step 2 — navigate and capture the first page:
1. `network requests --clear`
2. `navigate "$URL"`
3. `wait stable --timeout 25000` (timeout is normal on X; proceed)
4. `network requests --type xhr,fetch --filter TweetDetail` → take the latest entry's `request_id`
5. `network request <request_id>` → save full output to a file (e.g. `tmp/x-conversation-page-1.txt`)
6. `python scripts/parse-tweets.py --json-file tmp/x-conversation-page-1.txt --source tweet_detail` → emits JSON `{tweets, count, cursor_top, cursor_bottom}`. The focal tweet is the first element; subsequent elements are replies (in display order). Each reply carries `is_reply: true`, `in_reply_to_id`, `in_reply_to_user`, and `conversation_id == <root tweet id>`, which makes the reply tree reconstructable downstream.
Endpoint characteristic: URL contains `/i/api/graphql/<hash>/TweetDetail`. The query hash rotates; always filter by name.
Step 3 — paginate via scroll to load deeper replies and sub-replies:
1. `network requests --clear`
2. `scroll down --amount 5000`
3. `wait stable --timeout 10000`
4. `network requests --type xhr,fetch --filter TweetDetail` → newest entry's `request_id`
5. `network request <request_id>` → save to `tmp/x-conversation-page-N.txt`
6. `python scripts/parse-tweets.py --json-file tmp/x-conversation-page-N.txt --source tweet_detail`
Repeat Step 3 until any termination condition is met:
- Accumulated unique reply count reaches the user's target.
- `count == 0` on the current page.
- `cursor_bottom` is unchanged across two consecutive pages.
- The page shows a "Show more replies" button gated by visibility filters — `state` reveals it; the Agent may `click <index>` to expand more, then resume scrolling.
Error handling:
- If the focal tweet is deleted or protected, the response carries a `TweetTombstone` entry; `parse-tweets.py` filters tombstones, so the result will be `count: 0` — report to the user and stop.
- If no `TweetDetail` request appears after a scroll, wait 3 s and retry once. Persistent absence after two scrolls means the thread has been fully loaded.
- If a captcha challenge appears (visible in `state` as an authorization prompt), pause and ask the user.
Output example:
```json
{
"tweets": [
{
"type": "tweet",
"id": "2069990565530214798",
"url": "https://x.com/Rothmus/status/2069990565530214798",
"twitter_url": "https://twitter.com/Rothmus/status/2069990565530214798",
"text": "\"We are going to have a multi-racial nation in Singapore. ...\"",
"created_at": "Tue Jun 24 23:01:50 +0000 2026",
"lang": "en",
"source": "Twitter Web App",
"retweet_count": 91,
"reply_count": 84,
"like_count": 567,
"quote_count": 12,
"bookmark_count": 50,
"view_count": 64140,
"is_reply": false,
"is_retweet": false,
"is_quote": false,
"quote_id": null,
"quote_url": null,
"in_reply_to_id": null,
"in_reply_to_user": null,
"in_reply_to_user_id": null,
"conversation_id": "2069990565530214798",
"hashtags": [],
"mentions": [],
"urls": [],
"media": [],
"card": null,
"place": null,
"author": {
"id": "987654321",
"user_name": "Rothmus",
"name": "Rothmus",
"url": "https://x.com/Rothmus",
"is_verified": false,
"is_blue_verified": true,
"verified_type": null,
"profile_picture": "https://pbs.twimg.com/profile_images/.../photo.jpg",
"description": "Singapore observer ...",
"location": "Singapore",
"followers": 12345,
"following": 678,
"created_at": "Wed Jun 16 10:00:00 +0000 2021"
}
},
{
"type": "tweet",
"id": "2070027327409655973",
"url": "https://x.com/DinoLeadingNews/status/2070027327409655973",
"text": "@Rothmus @elonmusk Singapore proves that identity ...",
"is_reply": true,
"in_reply_to_id": "2069990565530214798",
"in_reply_to_user": "Rothmus",
"in_reply_to_user_id": "987654321",
"conversation_id": "2069990565530214798",
"like_count": 0,
"reply_count": 0,
"retweet_count": 0,
"quote_count": 0,
"bookmark_count": 0,
"view_count": 19,
"author": {"user_name": "DinoLeadingNews", "name": "Dino Leading News", "...": "..."},
"...": "..."
}
],
"count": 40,
"cursor_top": null,
"cursor_bottom": "DAACCgABHLoz..."
}
```
## Pagination
**Network Capture Pagination**: triggered by `scroll down`. X's page JS inserts the previous response's `cursor_bottom` into the next `TweetDetail` request's `variables.cursor`. Some sub-reply branches require clicking an in-page "Show replies" expander (`state` to locate, `click <index>`) before the next scroll surfaces them. Termination: `count == 0`, `cursor_bottom` does not advance across two consecutive pages, or user target reached.
## Success Criteria
`count >= 1 on the first page` (the focal tweet must be present unless the conversation root is deleted) AND `every tweet has non-null id, text, created_at, author.user_name, like_count, retweet_count, reply_count, conversation_id` AND `each reply tweet has is_reply == true and in_reply_to_id pointing to a tweet inside the thread`.
## Known Limitations
- Only replies visible to the logged-in session are returned; X enforces visibility filters (blocked accounts, soft-blocked replies, "Show additional replies, including those that may contain offensive content").
- Deleted root tweets return a tombstone; the Skill terminates with `count: 0`.
- Quote tweets that reference this conversation are NOT included — they live in a separate timeline; collect them via `x-tweet-search-by-query` with `filter:quote conversation_id:<id>` if needed.
- `view_count` is `null` for very new replies where X has not emitted `views.count`.
- Sustained polling triggers per-session throttling; stay under ~150 timeline calls per 15-minute window per session.
- The page may never reach `network-idle`; `wait stable` will frequently time out — proceed to read network anyway.
## Execution Efficiency
- **Batch orchestration**: write a bash script that iterates conversation ids serially in one session. For parallelism, fan out across multiple stealth browsers each with its own login.
- **Test before batch execution**: run one conversation end-to-end (page 1 + at least one paginated page) first.
- **Reduce redundant pre-operations**: keep the same session for many sequential conversations.
- **Error resumption**: persist `cursor_bottom` and accumulated reply IDs per conversation after every page.
- **De-duplicate by `id`**: the focal tweet may appear in multiple paginated responses (some endpoints repeat the focal tweet at the top of every page); merge by `id`.
## Experience Notes
Path: `{working-directory}/browser-act-skill-forge-memories/x-tweet-scraper-x-tweet-by-conversation.memory.md` (working directory is determined by the Agent running the Skill, typically the project root or current working directory)
**Before execution**: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.
**After execution**: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:
`{YYYY-MM-DD}: {what happened} → {conclusion}`
Normal execution does not write to the file. Do not record which conversation was scraped or how many replies were returned — those are task outputs, not experience.
don't have the plugin yet? install it then click "run inline in claude" again.