Onboard an agent to Bright Data. Use when a coding agent first encounters Bright Data — for live web work (search, scrape, structured data), for wiring Brigh...
---
name: agent-onboarding
description: |
Onboard an agent to Bright Data. Use when a coding agent first
encounters Bright Data — for live web work (search, scrape,
structured data), for wiring Bright Data into product code, for
installing the agent skill bundle, or for getting an API key. One
install command sets up the CLI, agent skills, and authentication.
Routes the reader to the right path: live tools, app integration,
MCP, auth-only, or direct REST without any install.
---
# Bright Data — Agent Onboarding
Bright Data gives agents reliable access to the open web: SERP results
that look like a real browser, clean markdown from any URL (with
CAPTCHA + JS handled), structured datasets for 40+ platforms (Amazon,
LinkedIn, Instagram, TikTok, YouTube, Reddit, Crunchbase, …), and a
Browser API for pages that need real interaction.
This skill is the entry point. Read it once, pick a path, then hand
off to the narrower skill that owns that path.
## Install
One command installs the CLI **and** the agent skills, and walks the
human through OAuth in the browser:
```bash
# macOS / Linux — fastest install
curl -fsSL https://cli.brightdata.com/install.sh | bash
# Cross-platform (or if you don't want the install script)
npm install -g @brightdata/cli
# One-off, no install
npx --yes --package @brightdata/cli brightdata <command>
```
Requires Node.js >= 20. After install, both `brightdata` and `bdata`
(shorthand) are available.
Then authenticate **once**:
```bash
bdata login
```
This single command:
1. Opens the browser for OAuth (or use `bdata login --device` on
headless / SSH machines)
2. Saves the API key locally — you never need to paste a token again
3. Auto-creates the required proxy zones (`cli_unlocker`,
`cli_browser`)
4. Sets sensible default configuration
For non-interactive setups you can pass the key directly:
```bash
bdata login --api-key <key>
# or
export BRIGHTDATA_API_KEY=<key>
```
Verify the install before doing real work:
```bash
bdata version
bdata config # confirms auth + zones
bdata zones # should list cli_unlocker, cli_browser
bdata budget # confirms account + balance
```
If any of these fail, route to Path C (auth) before continuing.
## Install agent skills (optional, recommended)
The CLI ships an installer that drops Bright Data skills directly into
your coding agent's skill directory:
```bash
# Interactive picker — choose skills + target agent
bdata skill add
# Install a specific skill
bdata skill add scrape
bdata skill add data-feeds
bdata skill add competitive-intel
# See everything available
bdata skill list
```
These are the skills you'll hand off to from the paths below
(`scrape`, `search`, `data-feeds`, `scraper-builder`,
`brightdata-cli`, `bright-data-mcp`, …).
## Choose your path
All paths share the same install + auth above. The difference is what
you do next.
| Situation | Path |
|---|---|
| Need web data **during this session** | **Path A** — live CLI tools |
| Need to **add Bright Data to app code** | **Path B** — SDK / REST integration |
| Want a **drop-in tool layer for an LLM agent** | **Path M** — MCP server |
| Need an **API key first** | **Path C** — auth only |
| Don't want to install anything | **Path D** — REST API directly |
If your task spans paths, do them in order: auth → live tools to
explore → app integration once the shape is known.
---
## Path A — Live web tools (CLI)
Use this when the agent itself needs web data right now: discovering
URLs, fetching clean content, pulling structured records from a known
platform, or running a quick competitive scan.
After install + login, hand off to the narrower skills:
- `brightdata-cli` — overall command surface (`scrape`, `search`,
`pipelines`, `status`, `zones`, `budget`, `config`)
- `search` — discovery via `bdata search` (Google / Bing / Yandex
SERP, structured JSON)
- `scrape` — clean content from a known URL via `bdata scrape`
(markdown / HTML / JSON / screenshot)
- `data-feeds` — structured records from 40+ supported platforms via
`bdata pipelines <type>` (Amazon, LinkedIn, Instagram, TikTok,
YouTube, Reddit, Crunchbase, Google Maps, …)
- `competitive-intel` — packaged competitor / pricing / review /
hiring / SEO analyses on top of the CLI
- `seo-audit` — sitemap-stratified live SEO audits
Default flow for live web work:
1. **Search first** when you need discovery
`bdata search "query" --json`
2. **Pipelines next** if the target is a supported platform — you get
structured JSON with no parsing
`bdata pipelines amazon_product "https://amazon.com/dp/..."`
3. **Scrape** when you have a URL and no platform pipeline applies
`bdata scrape "https://example.com" -f markdown`
4. **Browser API** only when the page truly needs clicks, forms, or
login (see the `brightdata-cli` skill for `bdata browser` and the
`bright-data-best-practices` browser-api reference)
When the task shifts from "fetch data now" to "wire this into an
app," switch to Path B.
---
## Path B — Integrate Bright Data into an app
Use this when you're building an application, agent, or workflow that
calls Bright Data from code and needs `BRIGHTDATA_API_KEY` (and a
zone) in `.env` or runtime config.
The required question on this path is:
> **What should Bright Data do in the product?**
Use the answer to pick the API:
| Job in product | API | Skill |
|---|---|---|
| Fetch a single page as markdown / HTML / JSON | Web Unlocker | `bright-data-best-practices` → `web-unlocker.md` |
| Search engine results in structured JSON | SERP API | `bright-data-best-practices` → `serp-api.md` |
| Structured records from supported platforms | Web Scraper API | `bright-data-best-practices` → `web-scraper-api.md` |
| JS-heavy / interactive pages with Playwright/Puppeteer | Browser API | `bright-data-best-practices` → `browser-api.md` |
| Build a custom scraper for an arbitrary site | All four, picked by site shape | `scraper-builder` |
### Pick a stack
- **Python** → use the official SDK
```bash
pip install brightdata-sdk
```
Hand off to `python-sdk-best-practices` for client setup
(async/sync), platform scrapers, SERP, datasets, Browser API, and
error handling.
- **Node / TypeScript / shell / other** → call the REST API directly
(Path D below has the endpoints), or use the CLI as a library via
`npx @brightdata/cli`.
- **LLM tool layer (Claude, ChatGPT, etc.)** → use the MCP server
(Path M).
### Set credentials
```dotenv
BRIGHTDATA_API_KEY=...
BRIGHTDATA_UNLOCKER_ZONE=cli_unlocker # created automatically by `bdata login`
BRIGHTDATA_SERP_ZONE=cli_unlocker # or a dedicated SERP zone
```
If you don't have a key yet, do Path C first.
### Smoke test before writing real code
Always run one real Bright Data request before scaling up integration
work — catches auth, zone, and quota issues before they hide inside
your app's error paths.
```bash
# Web Unlocker via REST
curl -sS https://api.brightdata.com/request \
-H "Authorization: Bearer $BRIGHTDATA_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com",
"zone": "'"$BRIGHTDATA_UNLOCKER_ZONE"'",
"format": "raw",
"data_format": "markdown"
}' | head -40
```
If this prints clean markdown, you're wired up. If not, check the
zone name and key.
---
## Path M — MCP server (LLM tool layer)
Use this when the consumer is an LLM agent that should call Bright
Data as tools (e.g., Claude Code, ChatGPT desktop, custom agent
loops). The MCP server exposes 60+ tools — search, scrape, structured
data per platform, browser automation — over a single URL.
Connect with:
```
https://mcp.brightdata.com/mcp?token=YOUR_BRIGHTDATA_API_TOKEN
```
Optional URL parameters:
| Parameter | Effect |
|---|---|
| `pro=1` | Enable all 60+ Pro tools |
| `groups=<name>` | Enable a tool group (`social`, `ecommerce`, `business`, `finance`, `research`, `app_stores`, `travel`, `browser`, `advanced_scraping`) |
| `tools=<names>` | Enable a specific tool list, comma-separated |
Hand off to the `bright-data-mcp` skill for tool selection, tool-group
auto-enabling, and workflow patterns. That skill explicitly replaces
WebFetch / WebSearch with Bright Data MCP equivalents.
---
## Path C — Get an API key (auth only)
Use this when the human still needs to sign up, sign in, or generate
a key. Skip this path if `bdata config` already shows an authenticated
account, or if `BRIGHTDATA_API_KEY` is already set in the environment.
### Easiest: use the CLI's OAuth flow
```bash
bdata login # browser-based OAuth
bdata login --device # headless / SSH (device-code flow)
```
This handles signup-or-signin, key generation, zone creation, and
local config in one step. Prefer this over manual flows.
### Manual: dashboard
If the human prefers the web UI:
1. Go to https://brightdata.com/cp (sign up if needed)
2. Create a **Web Unlocker zone** ("Add" → "Unlocker zone")
3. Copy the API key from the dashboard
4. Save it where the rest of the app reads secrets:
```bash
echo "BRIGHTDATA_API_KEY=..." >> .env
echo "BRIGHTDATA_UNLOCKER_ZONE=<zone-name>" >> .env
```
### Verify
```bash
bdata budget # any successful response means the key works
```
If verification fails, the key is wrong, the zone is wrong, or the
account has no active subscription — surface the error to the human
rather than guessing.
---
## Path D — Use Bright Data without installing anything
Use this when the environment can't run `npm` / `curl | bash`, or
when you only need one or two requests and don't want the CLI / SDK.
Works for both live agent work and app integration.
You still need an API key and a zone. Two ways to get them:
- **Human pastes it in** — if a key already exists, set
`BRIGHTDATA_API_KEY=...` and `BRIGHTDATA_UNLOCKER_ZONE=...` in the
environment
- **Browser flow** — do Path C; the dashboard issues both
**Base URL:** `https://api.brightdata.com`
**Auth header:** `Authorization: Bearer $BRIGHTDATA_API_KEY`
### Core endpoints
```http
# Web Unlocker — clean content from any URL
POST /request
{
"url": "https://target.com",
"zone": "<unlocker-zone>",
"format": "raw",
"data_format": "markdown" // or "html", "screenshot", "parsed_light"
}
```
```http
# SERP API — structured search results
# Use the same /request endpoint with a SERP zone and a search URL,
# adding `brd_json=1` to receive parsed JSON instead of raw HTML.
POST /request
{
"url": "https://www.google.com/search?q=web+scraping&brd_json=1",
"zone": "<serp-zone>",
"format": "raw"
}
```
```http
# Web Scraper API — structured data for 40+ platforms (async)
POST /datasets/v3/trigger?dataset_id=<id>
[ { "url": "https://amazon.com/dp/B09V3KXJPB" } ]
# then poll
GET /datasets/v3/snapshot/<snapshot_id>?format=json
```
For the full parameter surface (special headers like
`x-unblock-expect`, async response IDs, dataset progress states,
Browser API CDP commands), read the `bright-data-best-practices`
skill — its references are the source of truth for REST-level work.
### Documentation
- Product docs: https://docs.brightdata.com
- LLM-friendly docs index: https://docs.brightdata.com/llms.txt
- Dashboard (zones, keys, billing): https://brightdata.com/cp
---
## After onboarding — where to go next
Once the agent is set up, route the work to the narrowest skill that
fits. Quick map:
| User says… | Skill |
|---|---|
| "scrape this URL" / "get this page" | `scrape` |
| "search Google for…" / "find URLs about…" | `search` |
| "get Amazon / LinkedIn / Instagram / TikTok / YouTube / Reddit data" | `data-feeds` |
| "build a scraper for <site>" | `scraper-builder` |
| "analyze my competitor" / "compare pricing" | `competitive-intel` |
| "audit SEO" / "rank check" / "schema check" | `seo-audit` |
| "write Bright Data code in Python" | `python-sdk-best-practices` |
| "plug Bright Data into my LLM agent" | `bright-data-mcp` |
| "use the CLI" / "run from terminal" | `brightdata-cli` |
| "debug a Browser API session" | `brd-browser-debug` |
When in doubt, prefer the more specific skill: `data-feeds` over
`scrape` for supported platforms, `scraper-builder` over `scrape` for
multi-page extraction, `bright-data-mcp` over `brightdata-cli` when
the consumer is an LLM agent rather than a human at a terminal.
don't have the plugin yet? install it then click "run inline in claude" again.
Bright Data gives agents reliable access to the open web: SERP results that look like a real browser, clean markdown from any URL (CAPTCHA and JS handled), structured datasets for 40+ platforms (Amazon, LinkedIn, Instagram, TikTok, YouTube, Reddit, Crunchbase, etc.), and a Browser API for pages needing real interaction.
This skill is the entry point. read it once, pick a path, hand off to the narrower skill that owns that path.
onboard a coding agent to Bright Data when it first encounters the platform, whether for live web work (search, scrape, structured data), app integration, MCP setup, auth-only flows, or direct REST calls. use this skill to understand what Bright Data does, install the CLI and agent skills in one command, authenticate via OAuth, and route to the right downstream skill based on the job. one command installs everything and handles browser-based login for interactive setups; headless and non-interactive alternatives exist for CI/CD and remote environments.
npm install -g path, optional if using curl | bash)BRIGHTDATA_API_KEY env var)external connections:
https://cli.brightdata.com (for login / signup)https://cli.brightdata.com/install.sh (curl | bash flow)npm install -g flow)https://api.brightdata.com (for smoke tests and REST calls)https://brightdata.com/cp (manual key / zone creation, billing, account management)choose one of three install methods based on environment:
input: Node.js >= 20, npm available, internet access.
method A: macOS / Linux with curl (fastest).
curl -fsSL https://cli.brightdata.com/install.sh | bash
this downloads and runs a shell script that installs the CLI, registers the brightdata and bdata commands, and optionally prompts to add the CLI to PATH.
method B: npm (cross-platform, explicit).
npm install -g @brightdata/cli
installs the package globally; brightdata and bdata commands available after install.
method C: one-off, no install.
npx --yes --package @brightdata/cli brightdata <command>
npx --yes --package @brightdata/cli bdata <command>
runs the CLI directly without installing; adds ~5 seconds to first execution. suitable for CI/CD, containers, or testing before committing to a global install.
output: bdata command available in PATH; bdata --version returns a version string.
input: bdata command available, internet access, Bright Data account (create one if needed at https://brightdata.com).
for interactive terminal (human at keyboard):
bdata login
this opens a browser window for OAuth (signup if no account), prompts to authorize, then returns to the terminal and saves the API key locally. skips any need to paste tokens by hand.
for headless / SSH / CI/CD (no browser):
bdata login --device
displays a device code and URL (e.g., https://brightdata.com/device?code=ABC123). human opens this URL in any browser, enters the code, authorizes, and the CLI receives the token. suitable for remote servers and automation.
for non-interactive environment with key already known:
bdata login --api-key <key>
or set the env var and skip login entirely:
export BRIGHTDATA_API_KEY=<your-key>
output after successful login:
~/.brightdata/config.json or ~/.config/brightdata/config.json on Linux)BRIGHTDATA_API_KEY env var optionally set (human can add to .bashrc / .zshrc / .env if desired)cli_unlocker and cli_browserinput: bdata command available, login completed.
run three smoke tests:
bdata version
prints CLI version; confirms bdata is in PATH and executable.
bdata config
prints current config, authenticated account email, and active zones. if this fails, auth is broken; skip to decision point 1.
bdata zones
lists all zones on the account. must include cli_unlocker and cli_browser (created by bdata login). if missing, login did not complete; skip to decision point 1.
bdata budget
displays account balance and subscription status. any successful response means the API key works and the account is active. no response or 401 error means key is invalid or account is inactive; skip to decision point 1.
output: all four commands succeed with no errors. agent can proceed to path selection (step 4).
input: installation and auth verified (step 3 passed).
read the table below and identify the closest match to the work ahead:
| situation | path | next steps |
|---|---|---|
| agent needs web data right now (discover URLs, fetch clean content, pull structured records) | Path A: live CLI tools | hand off to brightdata-cli, search, scrape, data-feeds, competitive-intel, or seo-audit |
| building an app or workflow that calls Bright Data from code | Path B: SDK / REST integration | choose stack (Python SDK, Node REST, etc.); hand off to stack-specific skill (python-sdk-best-practices, bright-data-best-practices, etc.) |
| LLM agent (Claude, ChatGPT, custom loop) should call Bright Data as tools | Path M: MCP server | hand off to bright-data-mcp; connect to the MCP URL with your API token |
| human still needs to sign up, generate a key, or verify auth | Path C: auth only | follow the manual or CLI flows in section "path C" below |
| environment cannot run npm / curl, or only one or two REST calls needed | Path D: REST API directly | use curl, Python requests, or any HTTP client with the endpoints in section "path D" below |
output: one path selected; proceeding to that section next.
input: bdata command available, path selected (step 4).
if using Path A (live CLI tools), install the agent-facing skills to your coding agent's skill directory:
bdata skill add
interactive picker; choose which skills to install and where to install them (detects common agent frameworks like Anthropic, LangChain, etc.).
or install a specific skill:
bdata skill add scrape
bdata skill add data-feeds
bdata skill add search
bdata skill add scraper-builder
bdata skill add competitive-intel
bdata skill add seo-audit
list all available skills:
bdata skill list
output: skill files (SKILL.md) copied to the agent's skill directory (e.g., ~/.agents/skills/, or the agent framework's default location). agent can now call these skills as instructions.
decision point 1: smoke tests fail (version, config, zones, or budget command fails or returns 401 / 403).
if bdata config fails or shows "no authenticated account", auth did not persist. this can happen if:
branch A: retry interactive login.
bdata login
if this succeeds, re-run step 3 smoke tests. if it fails, move to branch B.
branch B: use non-interactive login with a known key. if the human has an existing Bright Data API key (from the dashboard at https://brightdata.com/cp), skip the browser flow:
bdata login --api-key <key>
export BRIGHTDATA_API_KEY=<key>
re-run smoke tests. if still failing, the key is wrong; ask the human to verify the key in the dashboard or create a new one.
branch C: route to Path C (auth only). if both branches fail, the human is not ready to use Bright Data yet. skip ahead to section "path C" below and follow the manual dashboard flow or troubleshoot the account status.
decision point 2: user needs web data now vs. building an app.
this determines whether to use Path A (live CLI, immediate results) or Path B (app integration, code-based, slower setup but durable).
if agent is exploring or discovering data (one-off requests, ad-hoc searches, testing): use Path A. the CLI is fastest; results come back in seconds; no coding required.
if agent is building a product, workflow, or service that will call Bright Data many times: use Path B. set up the SDK or REST client once; reuse it across many requests.
if both apply (explore first, then integrate): do Path A first, confirm the data shape and quality, then move to Path B once the integration is clear.
decision point 3: environment cannot run Node.js / npm.
if npm is not available or Node.js is too old, the CLI installation fails.
branch A: use Path D (REST API). call the REST endpoints directly with curl, Python, or any HTTP client. does not require npm or Node.js. requires an API key and zone (from Path C if needed).
branch B: use a container or CI/CD image with Node.js. if the agent runs in Docker or CI/CD, layer Node.js into the image before running the install. example Dockerfile:
FROM node:20-alpine
RUN npm install -g @brightdata/cli
RUN bdata login --api-key $BRIGHTDATA_API_KEY
ENTRYPOINT ["bdata"]
branch C: contact Bright Data support for alternative SDKs. if neither is feasible, Bright Data may offer SDKs or pre-built binaries in other languages; check https://docs.brightdata.com.
decision point 4: LLM agent (Claude, ChatGPT, custom loop) is the consumer.
if the downstream consumer of Bright Data is an LLM (not a human or CLI script), use the MCP server (Path M) instead of the CLI or REST directly.
why: MCP exposes 60+ Bright Data tools as a standard tool layer, compatible with Claude Code, ChatGPT, and custom agent loops. the LLM can call tools like brightdata_search, brightdata_scrape, and brightdata_get_social_data without invoking the CLI manually.
branch A: use the MCP server.
connect the LLM agent to https://mcp.brightdata.com/mcp?token=YOUR_BRIGHTDATA_API_TOKEN. hand off to bright-data-mcp skill for tool selection and workflow patterns.
branch B: fall back to REST or CLI if MCP is not supported. if the LLM framework does not support MCP, use Path B (SDK) or Path D (REST) and expose Bright Data calls as custom functions or tool definitions in the LLM's native tool format.
decision point 5: account has no active subscription or zero balance.
if bdata budget returns zero balance or "inactive subscription", the account cannot make requests.
branch A: purchase a subscription. visit https://brightdata.com/cp, choose a plan, and add payment. free tier is available with limits; paid tiers unlock higher volumes and more features.
branch B: request a trial or contact sales. for large deployments or special cases, contact Bright Data sales at https://brightdata.com/contact. they may offer a trial or custom arrangement.
branch C: use the browser API or MCP in read-only mode (if supported). some features may allow limited queries without active balance; check the feature-specific docs at https://docs.brightdata.com.
success: agent is onboarded and ready to use Bright Data. the following conditions are true:
bdata --version returns a semantic version (e.g., 2.1.3).bdata config prints an authenticated account email and no errors.bdata zones lists at least cli_unlocker and cli_browser.bdata budget returns account balance and subscription status (non-zero balance and active subscription preferred).bdata skill list shows one or more installed skills and no errors.format of the output: plain text console output from bdata commands; no file is created by this skill. config is stored in the user's home directory (e.g., ~/.brightdata/config.json).
file locations:
~/.brightdata/config.json (Unix/Mac) or %APPDATA%\brightdata\config.json (Windows)~/.agents/skills/ for Anthropic, ./skills/ for custom agents.env, .bashrc, .zshrc, or CI/CD secret storedata format: JSON for config and budget queries; plain text for version and zones. MCP server (Path M) returns JSON-RPC 2.0 over WebSocket.
the human or agent knows the skill worked when:
all smoke tests pass with no errors:
bdata version prints a version numberbdata config shows an email address (authenticated) and exits cleanlybdata zones lists cli_unlocker and cli_browserbdata budget shows account balance and subscription statusthe next downstream skill is clear: the human has read the path descriptions (Path A-D) and knows which section to hand off to. they can state: "we're doing Path A (live CLI)" or "Path B (SDK integration)" with confidence.
optional but confirming: a test request succeeds. if the human runs one of the examples from Path A, B, or D (e.g., bdata search "web scraping" --json or a REST curl call), the response comes back with data and no auth errors.
no further CLI debugging is needed. if any of the smoke tests fail, the decision point section above tells the human exactly which branch to take (retry login, use --api-key, check account status, etc.). the human does not have to guess.
credentials are safe and do not appear in logs or terminal history. the bdata login flow saves the API key to a local config file, not to the terminal or environment by default. if the human explicitly exports BRIGHTDATA_API_KEY, they understand they are doing so and manage it accordingly.
once the agent is set up, route the work to the narrowest skill that fits:
| user says | hand off to |
|---|---|
| "scrape this URL" / "get this page as markdown" | scrape |
| "search Google for…" / "find URLs about…" | search |
| "get Amazon / LinkedIn / Instagram / TikTok / YouTube / Reddit / Crunchbase data" | data-feeds |
| "build a scraper for |
scraper-builder |
| "analyze my competitor" / "compare pricing" / "review analysis" | competitive-intel |
| "audit SEO" / "rank check" / "schema check" / "backlink audit" | seo-audit |
| "write Bright Data code in Python" | python-sdk-best-practices |
| "write Bright Data code in Node / TypeScript" | bright-data-best-practices |
| "plug Bright Data into my LLM agent (Claude, ChatGPT, etc.)" | bright-data-mcp |
| "use the CLI" / "run Bright Data from terminal" / "orchestrate via shell" | brightdata-cli |
| "debug a Browser API session" / "troubleshoot Playwright / Puppeteer" | brd-browser-debug |
prefer the specific skill over the general one: data-feeds over scrape for supported platforms, scraper-builder over scrape for multi-page extraction, bright-data-mcp over brightdata-cli when the consumer is an LLM agent.
use this when the agent itself needs web data right now. after install and login, hand off to:
brightdata-cli , overall command surface (scrape, search, pipelines, status, zones, budget, config)search , discovery via bdata search (Google, Bing, Yandex SERP, structured JSON)scrape , clean content from a known URL via bdata scrape (markdown, HTML, JSON, screenshot)data-feeds , structured records from 40+ supported platforms via `bdata