Desktop Control (Windows)

SKILL.md

---
name: desktop-control
description: Control desktop applications on Windows — launch, close, focus, resize, move windows, simulate keyboard/mouse input, manage processes, control VSCode, read clipboard, and capture screen info. Use when the user wants to interact with any running program, switch windows, type text, press shortcuts, open files in VSCode, manage running processes, or get system display information.
---

# Desktop Control — Full Windows Application Control

## Publish-only note (ClawHub)
This Publish package includes scripts as `.ps1.txt` because Publish only accepts text files.
After download, rename each `*.ps1.txt` to `*.ps1` and place them in a `scripts/` folder to use the skill.

Control any desktop application on this Windows machine. Launch programs, manage windows, simulate input, control VSCode, and monitor processes — all via PowerShell scripts.

## CRITICAL: Script Location

All scripts are located relative to this skill folder:

```
SKILL_DIR = ~/.openclaw/workspace/skills/desktop-control/scripts
```

When running scripts, always use the full path:
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/<script>.ps1" -Action <action> [params]
```

## IMPORTANT: Safety Rules

1. **Before closing windows** — Ask user for confirmation if the window might have unsaved work
2. **Before killing processes** — Always confirm with user unless they explicitly asked to kill it
3. **Before sending input** — Make sure the correct window is focused first
4. **Clipboard** — Warn user if you are overwriting clipboard content

---

## Action Reference

### 1. Window Management (`app-control.ps1`)

Manage application windows — launch, close, focus, resize, move, snap.

#### List all visible windows
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action list-windows
```
Returns: PID, window title, position (X,Y), size (W×H), state (Normal/Minimized/Maximized)

#### Launch an application
```powershell
# By name (searches PATH and common locations)
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action launch -Target "notepad"

# By full path
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action launch -Target "C:\Program Files\MyApp\app.exe"

# With arguments
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action launch -Target "code" -Arguments "C:\Users\ibach\project"
```

#### Focus (bring to foreground)
```powershell
# By window title (partial match)
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action focus -Target "Visual Studio Code"

# By PID
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action focus -ProcId 12345
```

#### Close a window gracefully
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action close -Target "Notepad"
```

#### Minimize / Maximize / Restore
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action minimize -Target "Visual Studio Code"
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action maximize -Target "Visual Studio Code"
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action restore -Target "Visual Studio Code"
```

#### Move a window
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action move -Target "Notepad" -X 100 -Y 200
```

#### Resize a window
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action resize -Target "Notepad" -Width 800 -Height 600
```

#### Snap a window (half-screen)
```powershell
# Options: left, right, top, bottom, topleft, topright, bottomleft, bottomright
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/app-control.ps1" -Action snap -Target "Notepad" -Position left
```

---

### 2. Input Simulation (`input-sim.ps1`)

Simulate keyboard and mouse input into any application.

**IMPORTANT:** Always focus the target window FIRST using `app-control.ps1 -Action focus` before sending input.

#### Type text
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action type-text -Text "Hello, World!"
```

#### Send keyboard shortcut
```powershell
# Common shortcuts: Ctrl+S, Ctrl+C, Ctrl+V, Ctrl+Z, Alt+F4, Ctrl+Shift+P, Win+D
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action send-keys -Keys "Ctrl+S"
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action send-keys -Keys "Ctrl+Shift+P"
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action send-keys -Keys "Alt+Tab"
```

#### Send special keys
```powershell
# Keys: Enter, Tab, Escape, Backspace, Delete, Up, Down, Left, Right, Home, End, PageUp, PageDown, F1-F12
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action send-keys -Keys "Enter"
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action send-keys -Keys "F5"
```

#### Mouse click at coordinates
```powershell
# Left click
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-click -X 500 -Y 300

# Right click
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-click -X 500 -Y 300 -Button right

# Double click
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-click -X 500 -Y 300 -DoubleClick
```

#### Move mouse
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-move -X 500 -Y 300
```

#### Scroll
```powershell
# Scroll up (positive) or down (negative)
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-scroll -Clicks 3
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/input-sim.ps1" -Action mouse-scroll -Clicks -3
```

---

### 3. VSCode Control (`vscode-control.ps1`)

Control Visual Studio Code through the `code` CLI and extensions.

#### Open a file
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action open-file -Path "C:\Users\ibach\project\main.py"
```

#### Open a file at a specific line
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action goto -Path "C:\Users\ibach\project\main.py" -Line 42
```

#### Open a folder/workspace
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action open-folder -Path "C:\Users\ibach\project"
```

#### Open diff view
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action open-diff -Path "file1.py" -Path2 "file2.py"
```

#### List installed extensions
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action list-extensions
```

#### Install an extension
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action install-extension -ExtensionId "ms-python.python"
```

#### Uninstall an extension
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action uninstall-extension -ExtensionId "ms-python.python"
```

#### Open a new terminal in VSCode
```powershell
# This focuses VSCode and sends Ctrl+` to toggle terminal
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action new-terminal
```

#### Open VSCode command palette
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action command-palette
```

#### Run a VSCode command by name
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/vscode-control.ps1" -Action run-command -Command "workbench.action.toggleSidebarVisibility"
```

---

### 4. Process Management (`process-manager.ps1`)

Monitor and manage running processes.

#### List running processes
```powershell
# All processes
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action list

# Filter by name
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action list -Name "code"

# Top N by memory
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action list -SortBy memory -Top 10
```

#### Get detailed process info
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action info -ProcId 12345
```

#### Start a new process
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action start -Path "notepad.exe" -Arguments "C:\file.txt"
```

#### Kill a process (CONFIRM WITH USER FIRST)
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action kill -ProcId 12345
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action kill -Name "notepad"
```

#### Monitor process resource usage
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/process-manager.ps1" -Action monitor -ProcId 12345 -Duration 10
```

---

### 5. Screen & System Info (`screen-info.ps1`)

Get display information, window details, clipboard, and screenshots.

#### List displays/monitors
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action displays
```

#### Get active (focused) window info
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action active-window
```

#### Get detailed window info
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action window-info -Target "Visual Studio Code"
```

#### Take a screenshot
```powershell
# Full screen
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action screenshot -OutputPath "$HOME/screenshot.png"

# Specific window
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action screenshot -Target "Notepad" -OutputPath "$HOME/notepad-screenshot.png"
```

#### Read clipboard
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action clipboard-get
```

#### Set clipboard text
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action clipboard-set -Text "Text to copy"
```

#### Get system info (uptime, OS, resolution)
```powershell
powershell -ExecutionPolicy Bypass -File "$HOME/.openclaw/workspace/skills/desktop-control/scripts/screen-info.ps1" -Action system-info
```

---

## Common Workflows

### Open a file in VSCode and navigate to a specific line
```
1. vscode-control.ps1 -Action goto -Path "C:\path\to\file.py" -Line 42
```

### Type something into a specific application
```
1. app-control.ps1 -Action focus -Target "Notepad"
2. input-sim.ps1 -Action type-text -Text "Hello World"
```

### Save the current document in any app
```
1. app-control.ps1 -Action focus -Target "<app name>"
2. input-sim.ps1 -Action send-keys -Keys "Ctrl+S"
```

### Arrange two windows side-by-side
```
1. app-control.ps1 -Action snap -Target "Visual Studio Code" -Position left
2. app-control.ps1 -Action snap -Target "Chrome" -Position right
```

### Kill a frozen application
```
1. process-manager.ps1 -Action list -Name "frozen-app"
   (note the PID)
2. ASK USER FOR CONFIRMATION
3. process-manager.ps1 -Action kill -ProcId <pid>
```

### Take a screenshot of a specific window
```
1. screen-info.ps1 -Action screenshot -Target "Chrome" -OutputPath "$HOME/chrome.png"
```

---

## Error Handling

- If a script returns exit code 0 → success
- If a script returns exit code 1 → error (check stderr output for details)
- If a window is not found → try `list-windows` first to get the exact title
- If `code` CLI is not found → VSCode may not be in PATH; try launching it first

## Troubleshooting

- **"Window not found"** → Use `list-windows` to see exact window titles, then match more precisely
- **"Access denied"** → Some system processes need admin rights; inform the user
- **Input not working** → Make sure the target window is focused AND in the foreground
- **VSCode CLI not found** → Try `code --version` first; if missing, launch VSCode from Start Menu