Cloud browser automation CLI for AI agents powered by Scrapeless. Use when the user needs to interact with websites using cloud browsers, including navigatin...
SKILL.md

---
name: scrapeless-scraping-browser
description: Cloud browser automation CLI for AI agents powered by Scrapeless. Use when the user needs to interact with websites using cloud browsers, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task with residential proxies and anti-detection features. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "use a proxy", "bypass detection", or any task requiring cloud browser automation.
allowed-tools: Bash(npx scrapeless-scraping-browser-skills scrapeless-scraping-browser:*), Bash(scrapeless-scraping-browser:*)
---

# Cloud Browser Automation with scrapeless-browser

## Important: Session Management with --session-id

**All browser operation commands support the `--session-id` parameter to specify which Scrapeless session to use.**

### Recommended Workflow

```bash
# Step 1: Create a session and save the session ID
SESSION_ID=$(scrapeless-scraping-browser new-session --name "workflow" --ttl 1800 --json | jq -r '.taskId')

# Step 2: Use the session ID for all operations
scrapeless-scraping-browser --session-id $SESSION_ID open https://example.com
scrapeless-scraping-browser --session-id $SESSION_ID snapshot -i
scrapeless-scraping-browser --session-id $SESSION_ID click @e1

# Step 3: Close when done
scrapeless-scraping-browser --session-id $SESSION_ID close
```

### Automatic Session Management

If you don't specify `--session-id`:
1. The CLI will query for running sessions
2. If a running session exists, it will use the latest one
3. If no running session exists, it will create a new one automatically

**For production workflows, always use `--session-id` to ensure consistency.**

## Authentication Setup

Before using scrapeless-browser, you MUST set up authentication:

```bash
# Method 1: Config file (recommended, persistent)
scrapeless-scraping-browser config set apiKey your_api_token_here

# Method 2: Environment variable
export SCRAPELESS_API_KEY=your_api_token_here

# Verify it's set
scrapeless-scraping-browser config get apiKey
```

Get your API token from https://app.scrapeless.com

## Session Management Behavior

The CLI manages Scrapeless sessions with the following behavior:

- **Session Creation**: First command creates a new Scrapeless session
- **Session Persistence**: Sessions remain active only while connection is maintained
- **Session Termination**: Sessions automatically terminate when connection closes
- **Reconnection Limitation**: Cannot reconnect to terminated sessions

**Important**: For multi-step workflows, consider using the TypeScript API to maintain persistent connections.

## Core Workflow

Every browser automation follows this pattern:

1. **Create Session**: Create a session and save the session ID
2. **Navigate**: Use `--session-id` to navigate to URL
3. **Snapshot**: Get element refs with `--session-id`
4. **Interact**: Use refs to click, fill, select with `--session-id`
5. **Re-snapshot**: After navigation or DOM changes, get fresh refs

```bash
# Set API token first
scrapeless-scraping-browser config set apiKey your_token

# Create session
SESSION_ID=$(scrapeless-scraping-browser new-session --name "form-fill" --ttl 600 --json | jq -r '.taskId')

# Start automation with session ID
scrapeless-scraping-browser --session-id $SESSION_ID open https://example.com/form
scrapeless-scraping-browser --session-id $SESSION_ID snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"

scrapeless-scraping-browser --session-id $SESSION_ID fill @e1 "user@example.com"
scrapeless-scraping-browser --session-id $SESSION_ID fill @e2 "password123"
scrapeless-scraping-browser --session-id $SESSION_ID click @e3
scrapeless-scraping-browser --session-id $SESSION_ID wait --load networkidle
scrapeless-scraping-browser --session-id $SESSION_ID snapshot -i  # Check result
```

## Command Chaining

Commands can be chained with `&&` in a single shell invocation:

```bash
# Chain open + wait + snapshot
scrapeless-scraping-browser open https://example.com && scrapeless-scraping-browser wait --load networkidle && scrapeless-scraping-browser snapshot -i

# Chain multiple interactions
scrapeless-scraping-browser fill @e1 "user@example.com" && scrapeless-scraping-browser fill @e2 "password123" && scrapeless-scraping-browser click @e3
```

**When to chain:** Use `&&` when you don't need to read intermediate output. Run commands separately when you need to parse output first (e.g., snapshot to discover refs, then interact).

## Essential Commands

**Note**: All commands below support the optional `--session-id <id>` parameter.

```bash
# Navigation & Session
scrapeless-scraping-browser new-session [options]              # Create new browser session
scrapeless-scraping-browser [--session-id <id>] open <url>      # Navigate to URL
scrapeless-scraping-browser [--session-id <id>] close           # Close browser session
scrapeless-scraping-browser sessions                           # List running sessions
scrapeless-scraping-browser stop <taskId>                      # Stop specific session
scrapeless-scraping-browser stop-all                           # Stop all sessions
```

### Session Creation with Advanced Options

The `new-session` command supports extensive customization options:

```bash
# Basic session creation
scrapeless-scraping-browser new-session --name "my-session" --ttl 1800

# Session with proxy settings
scrapeless-scraping-browser new-session \
  --name "proxy-session" \
  --proxy-country US \
  --proxy-state CA \
  --proxy-city "Los Angeles" \
  --ttl 3600

# Session with custom browser configuration
scrapeless-scraping-browser new-session \
  --name "mobile-session" \
  --platform iOS \
  --screen-width 375 \
  --screen-height 812 \
  --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X)" \
  --timezone "America/Los_Angeles" \
  --languages "en,es"

# Session with recording enabled
scrapeless-scraping-browser new-session \
  --name "recorded-session" \
  --recording true \
  --ttl 7200
```

**Available Options:**
- `--name <name>`: Session name for identification
- `--ttl <seconds>`: Session timeout in seconds (default: 180)
- `--recording <true|false>`: Enable session recording
- `--proxy-country <code>`: Proxy country code (e.g., AU, US, GB, CN, JP)
- `--proxy-state <state>`: Proxy state/region (e.g., NSW, CA, NY, TX)
- `--proxy-city <city>`: Proxy city (e.g., sydney, newyork, london, tokyo)
- `--user-agent <ua>`: Custom user agent string
- `--platform <platform>`: Platform (Windows, macOS, Linux, iOS, Android)
- `--screen-width <px>`: Screen width in pixels (default: 1920)
- `--screen-height <px>`: Screen height in pixels (default: 1080)
- `--timezone <tz>`: Timezone (default: America/New_York)
- `--languages <langs>`: Comma-separated language codes (default: en)

```bash

# Snapshot
scrapeless-scraping-browser [--session-id <id>] snapshot -i             # Interactive elements with refs (recommended)
scrapeless-scraping-browser [--session-id <id>] snapshot -i -C          # Include cursor-interactive elements
scrapeless-scraping-browser [--session-id <id>] snapshot -s "#selector" # Scope to CSS selector

# Interaction (use @refs from snapshot)
scrapeless-scraping-browser [--session-id <id>] click @e1               # Click element
scrapeless-scraping-browser [--session-id <id>] fill @e2 "text"         # Clear and type text
scrapeless-scraping-browser [--session-id <id>] type @e2 "text"         # Type without clearing
scrapeless-scraping-browser [--session-id <id>] press Enter             # Press key
scrapeless-scraping-browser [--session-id <id>] scroll down 500         # Scroll page
scrapeless-scraping-browser [--session-id <id>] scroll down 500 --selector "div.content"  # Scroll within element

# Get information
scrapeless-scraping-browser [--session-id <id>] get text @e1            # Get element text
scrapeless-scraping-browser [--session-id <id>] get url                 # Get current URL
scrapeless-scraping-browser [--session-id <id>] get title               # Get page title
scrapeless-scraping-browser [--session-id <id>] screenshot              # Take screenshot
scrapeless-scraping-browser [--session-id <id>] screenshot --full       # Full page screenshot

# Wait
scrapeless-scraping-browser [--session-id <id>] wait @e1                # Wait for element
scrapeless-scraping-browser [--session-id <id>] wait --load networkidle # Wait for network idle
scrapeless-scraping-browser [--session-id <id>] wait --url "**/page"    # Wait for URL pattern
scrapeless-scraping-browser [--session-id <id>] wait 2000               # Wait milliseconds

# Cookies & Storage
scrapeless-scraping-browser [--session-id <id>] cookies                 # Get all cookies
scrapeless-scraping-browser [--session-id <id>] cookies set <name> <val> # Set cookie
scrapeless-scraping-browser [--session-id <id>] cookies clear           # Clear cookies
scrapeless-scraping-browser [--session-id <id>] storage local           # Get localStorage
scrapeless-scraping-browser [--session-id <id>] storage local set <k> <v>  # Set localStorage

# Multi-page
scrapeless-scraping-browser [--session-id <id>] pages                   # List all pages/tabs
scrapeless-scraping-browser [--session-id <id>] page <pageId>           # Switch to page
scrapeless-scraping-browser [--session-id <id>] tab new [url]           # Open new tab
scrapeless-scraping-browser [--session-id <id>] tab close [n]           # Close tab

# Live preview
scrapeless-scraping-browser live [taskId]                              # Get live preview URL
```
scrapeless-scraping-browser get url                 # Get current URL
scrapeless-scraping-browser get title               # Get page title
scrapeless-scraping-browser screenshot              # Take screenshot
scrapeless-scraping-browser screenshot --full       # Full page screenshot

# Wait
scrapeless-scraping-browser wait @e1                # Wait for element
scrapeless-scraping-browser wait --load networkidle # Wait for network idle
scrapeless-scraping-browser wait --url "**/page"    # Wait for URL pattern
scrapeless-scraping-browser wait 2000               # Wait milliseconds

# Cookies & Storage
scrapeless-scraping-browser cookies                 # Get all cookies
scrapeless-scraping-browser cookies set <name> <val> # Set cookie
scrapeless-scraping-browser cookies clear           # Clear cookies
scrapeless-scraping-browser storage local           # Get localStorage
scrapeless-scraping-browser storage local set <k> <v>  # Set localStorage

# Multi-page
scrapeless-scraping-browser pages                   # List all pages/tabs
scrapeless-scraping-browser page <pageId>           # Switch to page
scrapeless-scraping-browser tab new [url]           # Open new tab
scrapeless-scraping-browser tab close [n]           # Close tab

# Live preview
scrapeless-scraping-browser live                    # Get live preview URL
```

## Common Patterns

### Form Submission

```bash
scrapeless-scraping-browser config set apiKey your_token

scrapeless-scraping-browser open https://example.com/signup
scrapeless-scraping-browser snapshot -i
scrapeless-scraping-browser fill @e1 "Jane Doe"
scrapeless-scraping-browser fill @e2 "jane@example.com"
scrapeless-scraping-browser click @e3
scrapeless-scraping-browser wait --load networkidle
```

### Data Extraction

```bash
scrapeless-scraping-browser config set apiKey your_token

scrapeless-scraping-browser open https://example.com/products
scrapeless-scraping-browser snapshot -i --json
scrapeless-scraping-browser get text @e5 --json
```

### Common Session Configuration Scenarios

#### Mobile Device Simulation
```bash
# Simulate iPhone for mobile-specific content
SESSION_ID=$(scrapeless-scraping-browser new-session \
  --name "mobile-test" \
  --platform iOS \
  --screen-width 375 \
  --screen-height 812 \
  --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X)" \
  --json | jq -r '.taskId')

scrapeless-scraping-browser --session-id $SESSION_ID open https://m.example.com
```

#### Geographic Content Testing
```bash
# Access content from different regions
SESSION_ID=$(scrapeless-scraping-browser new-session \
  --name "geo-test" \
  --proxy-country AU \
  --proxy-city sydney \
  --timezone "Australia/Sydney" \
  --languages "en-AU,en" \
  --json | jq -r '.taskId')

scrapeless-scraping-browser --session-id $SESSION_ID open https://example.com
```

#### High-Resolution Desktop Testing
```bash
# Test on high-resolution displays
SESSION_ID=$(scrapeless-scraping-browser new-session \
  --name "desktop-4k" \
  --platform macOS \
  --screen-width 3840 \
  --screen-height 2160 \
  --json | jq -r '.taskId')

scrapeless-scraping-browser --session-id $SESSION_ID open https://example.com
```

#### Session Recording for Debugging
```bash
# Enable recording for troubleshooting
SESSION_ID=$(scrapeless-scraping-browser new-session \
  --name "debug-session" \
  --recording true \
  --ttl 7200 \
  --json | jq -r '.taskId')

scrapeless-scraping-browser --session-id $SESSION_ID open https://example.com
# Session recording will be available for review
```

### Session Persistence

**Important**: Scrapeless sessions terminate when connections close. For persistent workflows, use the TypeScript API:

```bash
scrapeless-scraping-browser config set apiKey your_token

# Create a session for login
scrapeless-scraping-browser create --name "login-session" --ttl 1800
scrapeless-scraping-browser open https://app.example.com/login
scrapeless-scraping-browser snapshot -i
scrapeless-scraping-browser fill @e1 "username"
scrapeless-scraping-browser fill @e2 "password"
scrapeless-scraping-browser click @e3
scrapeless-scraping-browser wait --url "**/dashboard"

# For subsequent operations, create a new session
# (Cannot reuse previous session due to connection termination)
scrapeless-scraping-browser create --name "dashboard-session" --ttl 1800
scrapeless-scraping-browser open https://app.example.com/dashboard
```

**Better Alternative**: Use TypeScript API for multi-step workflows:

```typescript
import { BrowserManager } from './dist/browser.js';

const manager = new BrowserManager();
await manager.launch({ id: 'persistent-workflow', action: 'launch' });

const page = manager.getPage();
await page.goto('https://app.example.com/login');
// Login persists throughout the session
await page.fill('#username', 'user');
await page.fill('#password', 'pass');
await page.click('#login');
await page.waitForURL('**/dashboard');
await page.goto('https://app.example.com/profile');
// Session and cookies maintained

await manager.close();
```

### Using Proxies

```bash
scrapeless-scraping-browser config set apiKey your_token

# Use residential proxy from specific country
scrapeless-scraping-browser config set proxyCountry US
scrapeless-scraping-browser open https://example.com

# Use custom proxy
scrapeless-scraping-browser config set proxyUrl "http://user:pass@proxy.com:8080"
scrapeless-scraping-browser open https://example.com

# Use proxy with state and city (v2 API)
scrapeless-scraping-browser config set proxyCountry US
scrapeless-scraping-browser config set proxyState CA
scrapeless-scraping-browser config set proxyCity "Los Angeles"
scrapeless-scraping-browser open https://example.com
```

### Browser Fingerprinting

```bash
scrapeless-scraping-browser config set apiKey your_token

# Set browser fingerprint to avoid detection
scrapeless-scraping-browser config set fingerprint chrome
scrapeless-scraping-browser open https://example.com

# Customize browser fingerprint details
scrapeless-scraping-browser config set userAgent "Mozilla/5.0 (iPhone; CPU iPhone OS 15_0 like Mac OS X)"
scrapeless-scraping-browser config set platform iOS
scrapeless-scraping-browser config set screenWidth 375
scrapeless-scraping-browser config set screenHeight 812
scrapeless-scraping-browser config set timezone "Asia/Shanghai"
scrapeless-scraping-browser config set languages "zh-CN,en"
scrapeless-scraping-browser open https://example.com
```

### Session Recording

```bash
scrapeless-scraping-browser config set apiKey your_token

# Enable session recording for debugging
scrapeless-scraping-browser config set sessionRecording true
scrapeless-scraping-browser open https://example.com
# ... perform actions ...
scrapeless-scraping-browser close
# Recording will be available in Scrapeless dashboard
```

### Multiple Sessions

**Note**: Due to session termination behavior, the `--session-id` parameter has limitations. For reliable multi-session workflows, create separate sessions for each task:

```bash
scrapeless-scraping-browser config set apiKey your_token

# Create first session for task A
scrapeless-scraping-browser create --name "task-a" --ttl 1800
scrapeless-scraping-browser open https://site-a.com
# Complete task A operations...

# Create second session for task B  
scrapeless-scraping-browser create --name "task-b" --ttl 1800
scrapeless-scraping-browser open https://site-b.com
# Complete task B operations...

# List all running sessions
scrapeless-scraping-browser sessions

# Stop specific session
scrapeless-scraping-browser stop <taskId>

# Stop all sessions
scrapeless-scraping-browser stop-all
```

**Alternative**: For complex multi-session workflows, use the TypeScript API which supports persistent connections.

## Configuration File

Configuration is managed via the `config` command. All settings are stored in `~/.scrapeless/config.json`.

**Priority**: Config file > Environment variable (only `SCRAPELESS_API_KEY` supports env var)

Available configuration options:
- `apiKey` - Your Scrapeless API token (required)
- `apiVersion` - API version (v1 or v2, default: v2)
- `sessionTtl` - Session timeout in seconds
- `sessionName` - Session name for identification
- `sessionRecording` - Enable session recording (true/false)
- `proxyUrl` - Custom proxy URL
- `proxyCountry` - Proxy country code
- `proxyState` - Proxy state/province
- `proxyCity` - Proxy city
- `fingerprint` - Browser fingerprint
- `debug` - Enable debug logging

## Agent Mode (JSON Output)

Use `--json` for machine-readable output:

```bash
scrapeless-scraping-browser snapshot -i --json
scrapeless-scraping-browser get text @e1 --json
scrapeless-scraping-browser is visible @e2 --json
```

## Ref Lifecycle (Important)

Refs (`@e1`, `@e2`, etc.) are invalidated when the page changes. Always re-snapshot after:

- Clicking links or buttons that navigate
- Form submissions
- Dynamic content loading

```bash
scrapeless-scraping-browser click @e5              # Navigates to new page
scrapeless-scraping-browser snapshot -i            # MUST re-snapshot
scrapeless-scraping-browser click @e1              # Use new refs
```

## Session Management

### Important Session Behavior

**Critical**: Scrapeless sessions have specific connection requirements:

- ✅ **Sessions work perfectly with persistent connections**
- ❌ **Sessions automatically terminate when the connection is closed**
- ❌ **Reconnecting to a terminated session will fail**

### Recommended Usage Patterns

#### For Single Operations
```bash
# Create and use a session for a single task
scrapeless-scraping-browser create --name "single-task" --ttl 600
scrapeless-scraping-browser open https://example.com
scrapeless-scraping-browser screenshot
```

#### For Multi-Step Operations
For complex workflows requiring multiple steps, use the TypeScript API instead of CLI:

```typescript
import { BrowserManager } from './dist/browser.js';

const manager = new BrowserManager();
await manager.launch({ id: 'workflow', action: 'launch' });

const page = manager.getPage();
await page.goto('https://example.com');
await page.screenshot({ path: 'step1.png' });
await page.goto('https://another-site.com');
await page.screenshot({ path: 'step2.png' });

await manager.close();
```

### Session ID Parameter Limitations

The `--session-id` parameter has limitations due to Scrapeless session behavior:

```bash
# This will fail if the session connection was previously closed
scrapeless-scraping-browser --session-id <session-id> open https://example.com
# Error: Session has been terminated and cannot be reconnected
```

**Workaround**: Create new sessions for each workflow instead of reusing session IDs.

### Session Management Commands

Always close sessions when done to avoid leaked resources:

```bash
scrapeless-scraping-browser close                    # Close current session
scrapeless-scraping-browser stop <taskId>            # Stop specific session by ID
scrapeless-scraping-browser stop-all                 # Stop all sessions
```

Check running sessions:

```bash
scrapeless-scraping-browser sessions
# Returns: sessionId, createdAt, status, sessionName
```

## Live Preview

Get a real-time preview of your browser session via WebSocket:

```bash
# Get live preview URL for current session
scrapeless-scraping-browser live

# Get live preview URL for specific session
scrapeless-scraping-browser live abc123def456

# Returns live preview URL for browser viewing
# Open this URL in your browser to view the live session
```

## Timeouts and Slow Pages

For slow websites, use explicit waits:

```bash
# Wait for network activity to settle
scrapeless-scraping-browser wait --load networkidle

# Wait for specific element
scrapeless-scraping-browser wait @e1

# Wait for URL pattern
scrapeless-scraping-browser wait --url "**/dashboard"

# Wait fixed duration (milliseconds)
scrapeless-scraping-browser wait 5000
```

## Error Handling

Common errors and solutions:

**Authentication Error:**
```bash
# Make sure API token is set
scrapeless-scraping-browser config set apiKey your_token
# Or use environment variable
export SCRAPELESS_API_KEY=your_token
```

**Session Not Found:**
```bash
# Session may have expired, create new one
scrapeless-scraping-browser open https://example.com
```

**Element Not Found:**
```bash
# Re-snapshot to get fresh refs
scrapeless-scraping-browser snapshot -i
```

**Timeout:**
```bash
# Increase session timeout (in seconds)
scrapeless-scraping-browser config set sessionTtl 600
scrapeless-scraping-browser open https://example.com
```

## Debugging

Enable debug mode for detailed logs:

```bash
scrapeless-scraping-browser config set debug true
scrapeless-scraping-browser open https://example.com
```

Or use `--debug` flag:

```bash
scrapeless-scraping-browser --debug open https://example.com
```

## Configuration Options

| Configuration | Description |
|---------------|-------------|
| `apiKey` | Your API token (required) |
| `apiVersion` | API version (v1 or v2, default: v2) |
| `sessionTtl` | Session timeout in seconds |
| `sessionName` | Session name for identification |
| `sessionRecording` | Enable session recording (true/false) |
| `proxyUrl` | Custom proxy URL |
| `proxyCountry` | Proxy country code |
| `proxyState` | Proxy state/province |
| `proxyCity` | Proxy city |
| `fingerprint` | Browser fingerprint |
| `userAgent` | Custom user agent string |
| `platform` | Platform type (Windows, Linux, macOS, iOS, Android) |
| `screenWidth` | Screen width in pixels |
| `screenHeight` | Screen height in pixels |
| `timezone` | Timezone (e.g., America/New_York, Asia/Shanghai) |
| `languages` | Comma-separated language codes (e.g., en,zh-CN) |
| `debug` | Enable debug logging |

Set configuration using:
```bash
scrapeless-scraping-browser config set <key> <value>
```

Or use environment variable for API key only:
```bash
export SCRAPELESS_API_KEY=your_token
```

## Key Differences from Local Browsers

1. **Cloud-based**: Runs on Scrapeless infrastructure, not locally
2. **Residential Proxies**: Built-in support for residential proxy rotation
3. **Anti-detection**: Automatic browser fingerprinting and stealth features
4. **Session Recording**: Optional recording of browser sessions
5. **No Installation**: No need to install Chrome/Chromium locally
6. **Scalable**: Run multiple sessions in parallel

## Limitations

- Profile management is not currently supported
- Browser extensions are not currently supported
- Requires active internet connection
- Requires valid Scrapeless API token

## Best Practices

1. **Always set API token** before running commands (via config or env var)
2. **Let automatic session management work** - the CLI will reuse sessions intelligently
3. **Use --session-id** only when you need parallel workflows
4. **Close sessions** when done to avoid charges
5. **Use config file** for persistent settings instead of environment variables
6. **Enable recording** for debugging complex flows
7. **Re-snapshot** after page changes
8. **Use --json** for programmatic access
9. **Set session timeout** appropriately for your use case (in seconds)

## Updates

Check for and install updates:

```bash
# Check version
scrapeless-scraping-browser version

# Update via npm
npm update -g scrapeless-scraping-browser-skills
```

## Support

- Documentation: https://docs.scrapeless.com
- API Reference: https://api.scrapeless.com/docs
- GitHub Issues: https://github.com/scrapeless-ai/scraping-browser-skill/issues
Scrapeless Scraping Browser Skill

SKILL.md

related skills
