MacOS Desktop Control (Mouse, Keyboard, Screenshots)

SKILL.md

# macOS Control Skill

A high-fidelity automation bridge for macOS (Darwin) that enables agents to perceive the desktop state and execute precise mouse and keyboard interactions.

## 🛠 Included Scripts
This skill leverages specialized wrappers located in the `/scripts` directory to interface with system-level binaries:

### 1. `cliclick_wrapper.sh`
A dedicated wrapper for the `cliclick` utility to handle synthetic input events.
- **Path**: `scripts/cliclick_wrapper.sh`
- **Logic**: Executes `/opt/homebrew/bin/cliclick` with passed arguments.
- **Capabilities**: Left/Right clicks, mouse movement, and keyboard emulation.

### 2. `vision_wrapper.sh`
The visual perception engine for the skill.
- **Path**: `scripts/vision_wrapper.sh`
- **Logic**: Utilizes the native macOS `screencapture` utility in silent mode (`-x`).
- **Output**: Generates a standard PNG at `/tmp/claw_view.png`.

---

## 🚀 Tool Specifications

### `see`
Captures the current screen state for visual analysis.
* **Returns**: A string confirming the filepath of the capture.
* **Use Case**: Identifies UI elements, window positions, and application states.

### `click`
Sends precise commands to the mouse and keyboard.
* **Usage**: `click "c:x,y"` (Click) or `click "m:x,y"` (Move).
* **Syntax**: Supports all `cliclick` standard notation including `w:` (wait) and `t:` (type).

---

## ⚙️ Requirements & Setup

1. **Binary Dependency**:
   ```bash
   brew install cliclick
   ```