Owl Browser

intent

owl browser is an ai-native browser that renders pages as owlmark: a compact, handle-addressable text view of on-screen content instead of screenshots or raw html. use this skill to navigate sites, observe rendered pages, and interact with elements via handle tokens rather than pixel coordinates or css selectors. owl is cheaper than screenshot-based automation and removes the guesswork of coordinate-based clicking. run this skill whenever you need to browse the web as an agent, fill forms, read text, or trigger page actions programmatically.

inputs

• OWL_API_ENDPOINT (env var, required): base url of the owl browser http api. examples: http://localhost:8080 (raw server) or http://localhost:80 (docker/nginx). set this before calling any owl tool.
• OWL_API_TOKEN (env var, required): bearer token for the owl browser api. must match OWL_HTTP_TOKEN on the server. set this before calling any owl tool.
• curl (binary, required): used to make http requests to the owl api.
• context_id (runtime parameter, per-call): a session identifier returned by browser_create_context. pass this to every subsequent tool call in the same browsing session.
• render_mode (optional, default "agent"): one of "agent" (owlmark text, minimal tokens), "both" (owlmark plus screenshot capability), or "pixel" (legacy human rendering). set to "both" if you will later call browser_screenshot to judge visual design.

all calls are POST $OWL_API_ENDPOINT/execute/<tool> with json body and Authorization: Bearer $OWL_API_TOKEN header.

procedure

1. create session: call browser_create_context with render_mode: "agent" (or "both" if you plan to screenshot later). capture result.context_id from the response. do not pass context_id to this call; the server generates it.

2. navigate: call browser_navigate with the context_id and target url. this does not return rendered content.

3. observe page: call browser_observe with the context_id. capture the render (owlmark text), handles (actionable element tokens like l5, x27), metadata (status, dropped surfaces), and token_estimate. inspect metadata.status: if "pending", wait for retry_after_ms milliseconds, then re-observe; if "incomplete", re-observe once, then use screenshot if content is still missing; if "ready", proceed to act.

4. act on handles: use one of the following based on desired action:

   • to click an element, call browser_click with context_id and selector (the handle token, e.g. "l5"). check the response effect field: "navigated" (page loaded), "dom-changed" (page modified), or "no-effect" (nothing happened).
   • to type text, call browser_type with context_id, selector, and text. optionally call browser_clear_input first if the field may have prior content.
   • to submit a form or trigger keyboard action, call browser_press_key with context_id and key name (e.g. "Enter", "Escape", "Tab").

5. re-observe after action: after every click, type, or key press, immediately call browser_observe again with the context_id to see the new page state and get fresh handles. do not assume an action succeeded until you have observed the result.

6. handle late-loading content: if a page is not fully rendered (status is "pending" or "incomplete"), the server will tell you via metadata.dropped_surfaces what content is missing (e.g. "canvas", "image:N", "sparse_main", "shell_unhydrated"). either retry observe, or switch render_mode to "both" and use browser_screenshot plus your own vision model to read the content.

7. zoom in on long pages: for reference or list-heavy pages, call browser_observe with detail: "outline" to get a headings-only map. once you identify the section you need, call browser_expand with the collapsed region handle to re-serialize it at full detail. alternatively, call browser_read_node with a specific element handle to extract its full text without screenshots.

8. close session: when done, call browser_close_context with the context_id. this frees the browser instance.

decision points

• if the page status is "pending": the page has not finished rendering its main content. the metadata includes reason and retry_after_ms. wait that duration and call observe again. do not treat a pending render as an empty or failed page.

• if the page status is "incomplete": chrome has rendered but main content did not load. re-observe once. if still incomplete, switch render_mode to "both" and use browser_screenshot plus your own vision.

• if metadata.dropped_surfaces includes canvas, webgl, or image elements: the page has visual content that owlmark cannot capture as text. enable render_mode "both" and use browser_screenshot to judge visual design or read image-based text.

• if metadata.dropped_surfaces includes sparse_main, shell_unhydrated, or main_content_unrendered: content is late or withheld (e.g. behind a client-side router). re-observe after a short delay. if still empty, use screenshot.

• if a click returns effect "no-effect": the element may not be interactive, or the action did not reach the server. re-observe and verify the page state. if the element is still present, try clicking again.

• if a click returns effect "scrolled": the link was an anchor (href="#section"). the page did not navigate; the viewport moved instead. this is expected. re-observe to see the scrolled content.

• if you receive STALE_HANDLE error: the handle token is no longer valid (the page changed or a navigation occurred). immediately re-observe to get fresh handles and retry.

• if a navigation click causes the browser to crash and auto-respawn (rare, on slow sites): the context may be lost. instead of retrying the click, recreate the context and call browser_navigate directly to the destination url.

• if the page contains a pdf or embedded plugin (canvas, webgl): you cannot interact with plugin controls via handles. use render_mode "both" and browser_screenshot to read the content visually.

• if api calls time out or receive rate-limit errors: the owl server may be overloaded or the api token may be invalid. check OWL_API_TOKEN and OWL_API_ENDPOINT; retry after a backoff delay (1 second minimum); if errors persist, restart the owl browser service.

output contract

each tool returns a json object with structure:

{
  "success": true or false,
  "result": { ... tool-specific data ... },
  "error": "error message if success is false"
}

key outputs from each tool:

• browser_create_context: result.context_id (string, e.g. "ctx_abc123def456"). store this for all later calls.

• browser_navigate: no rendered content. proceed to browser_observe.

• browser_observe:

  • result.render (string): owlmark text representation of the page.
  • result.handles (object): map of handle tokens to element metadata (e.g. {"l5": {"type": "link", "text": "Pricing"}, "x27": {"type": "textbox", "label": "Email", "value": ""}}).
  • result.metadata.status (string): "ready", "pending", or "incomplete".
  • result.metadata.dropped_surfaces (array): content types that could not be captured as text (e.g. ["canvas", "image:1"]).
  • result.metadata.retry_after_ms (number, if status is "pending"): milliseconds to wait before re-observing.
  • result.token_estimate (number): estimated tokens consumed by this render.

• browser_click / browser_type / browser_press_key:

  • result.effect (string): "navigated", "dom-changed", "scrolled", or "no-effect".
  • re-observe immediately after to see the result.

• browser_screenshot: result.image (base64-encoded png or similar). only call if render_mode is "both" or "pixel".

• browser_expand: re-serialized render of the expanded region at higher detail.

• browser_read_node: result.text (string): full text content of the element.

• browser_close_context: no returned data. the session is terminated.

all responses include success: true on success, success: false on error. if success is false, error contains a descriptive message (e.g. "STALE_HANDLE", "context not found", "invalid selector").

outcome signal

the skill has succeeded when:

• you have called browser_observe and received metadata.status: "ready" after a navigation or action, confirming the page is readable.
• you have extracted the information or triggered the action you needed. for example, you filled a form and the page showed a success message, or you clicked a link and reached the destination page.
• you called browser_close_context and received success: true, cleanly ending the session.

you have failed if:

• browser_observe returns metadata.status: "incomplete" after multiple retries and you cannot extract the content even with screenshot.
• a click or type returns STALE_HANDLE or "no-effect" repeatedly.
• the api returns success: false with an error message indicating the context is invalid or the browser crashed.
• a required handle is missing from the render (e.g. you expected to see a login button but it does not appear in the handles map).