OpenClaw Voice Control

intent

set up local macOS voice control for OpenClaw agents using wakeword activation, local ASR (FunASR / SenseVoice), TTS playback, and optional background runtime with launchd auto-start. use this skill when deploying voice input to a running local OpenClaw service, troubleshooting voice entrypoints, or managing background voice daemons. this is a local deployment skill, not a prompt-only helper.

inputs

prerequisites

• macOS (darwin only)
• python 3.11 or newer
• a running local OpenClaw service
• git and launchctl available
• microphone permission in system settings

external connections

OpenClaw service

• connection type: HTTP API
• env var: OPENCLAW_BASE_URL (e.g., http://127.0.0.1:18789/v1/chat/completions)
• auth env var: OPENCLAW_TOKEN (obtain from ~/.openclaw/openclaw.json under gateway key, not from ~/.openclaw/identity/device-auth.json)
• edge case: if service is unreachable or auth token is expired, voice control will fail to forward recognized text

model dependencies

• SenseVoice ASR model: specified by SENSEVOICE_MODEL_PATH env var. download via modelscope or provide local path
• VAD model: specified by SENSEVOICE_VAD_MODEL_PATH env var. auto-cached by FunASR on first run
• edge case: large model downloads may timeout on slow connections or fail if disk space is insufficient

secrets and configuration

required in .env

• OPENCLAW_BASE_URL
• OPENCLAW_TOKEN
• SENSEVOICE_MODEL_PATH
• SENSEVOICE_VAD_MODEL_PATH

default-route optional in .env

• WAKEWORD_PROVIDER (default: openwakeword)
• OPENWAKEWORD_MODEL_NAME (default: hey jarvis)
• OPENCLAW_AGENT_ID (default: main)
• OPENCLAW_MODEL (default: openclaw:main)
• OPENCLAW_USER (default: openclaw-voice-control)
• OPENWAKEWORD_THRESHOLD (tuning only, optional)
• VOICE_CONTROL_PYTHON_BIN (interpreter override, troubleshooting only)
• VOICE_CONTROL_OVERLAY_PYTHON_BIN (interpreter override, troubleshooting only)

porcupine-route optional (only if user explicitly chooses porcupine)

• PICOVOICE_ACCESS_KEY
• WAKEWORD_FILE (local .ppn file path)

repository source

• GitHub: https://github.com/CarrotYuan/openclaw-voice-control.git
• always clone into the current installed skill workspace, not a sibling directory

macOS TTS voice setup

• optional: if user wants to change TTS voice, that voice must be pre-downloaded in System Settings > Accessibility > Spoken Content > the i button next to System Voice > Language > Voice

procedure

phase 1: repository sync and workspace prep

1. ask for confirmation to fetch the repository into the current installed skill workspace. explain that this will run git init, git remote add, and git fetch --depth 1.

   input: user confirmation

   output: proceed to step 2 or stop

2. sync the repository from https://github.com/CarrotYuan/openclaw-voice-control.git into the current workspace using:

   git init
   git remote add origin https://github.com/CarrotYuan/openclaw-voice-control.git
   git fetch --depth 1 origin main
   git checkout -B main FETCH_HEAD
   

   input: network access, git credentials if needed

   output: workspace now contains scripts/, src/, config/, launchagents/, and README.md

   edge case: if git fails, stop and report the error; do not substitute an unrelated local directory

3. verify repository structure. confirm the workspace contains:

   • scripts/
   • src/
   • config/
   • launchagents/
   • README.md

   input: filesystem check

   output: if any file is missing, fail and tell user to re-sync; if all present, proceed

phase 2: python environment and dependencies

4. create and activate virtual environment. ask for confirmation before running pip install -e ..

   input: user confirmation

   output: proceed to step 5 or stop

5. set up python venv:

   python3 -m venv .venv
   source .venv/bin/activate
   

   input: python3 available, disk space for venv

   output: .venv/ directory created, shell prompt shows (.venv) indicator

   edge case: if python3 is missing or not python 3.11+, stop and tell user to upgrade

6. install the project in editable mode:

   pip install -e .
   

   input: active venv, network access for pip

   output: openclaw-voice-control and dependencies installed

   edge case: if install fails due to missing system headers or binary dependencies, report error and suggest brew install for common missing deps

phase 3: model downloads and configuration

7. download SenseVoice ASR model. ask for confirmation before downloading (large download, may take minutes). explain what will be downloaded and where.

   input: user confirmation, network access, ~10GB disk space

   output: proceed to step 8 or stop

8. download SenseVoice model:

   ./.venv/bin/modelscope download --model iic/SenseVoiceSmall --local_dir models/SenseVoiceSmall
   

   input: modelscope CLI, network access

   output: models/SenseVoiceSmall/ directory populated

   edge case: if download stalls or times out, user can resume by running the same command again; if disk is full, stop and free space

9. download VAD model. explain this will cache the FunASR VAD model locally.

   input: user confirmation, network access, ~500MB disk space

   output: proceed to step 10 or stop

10. download VAD model:

    ./.venv/bin/python - <<'PY'
    from funasr import AutoModel
    AutoModel(model='fsmn-vad', disable_update=True)
    PY
    

    input: active venv, network access

    output: VAD model cached in FunASR cache directory

    edge case: if network is offline, this will fail; inform user to retry when online

11. create .env file from template:

    cp .env.example .env
    

    input: .env.example must exist in repository

    output: .env file created in workspace root

12. populate required .env values. guide user through filling:

    • OPENCLAW_BASE_URL: get from running OpenClaw service (e.g., http://127.0.0.1:18789/v1/chat/completions)
    • OPENCLAW_TOKEN: get from ~/.openclaw/openclaw.json under gateway key
    • SENSEVOICE_MODEL_PATH: set to models/SenseVoiceSmall (relative path from workspace)
    • SENSEVOICE_VAD_MODEL_PATH: set to a valid VAD model path (FunASR will auto-cache if not specified, or user can provide explicit path)

    input: user provides real values

    output: .env contains all 4 required values

    edge case: if OPENCLAW_TOKEN is missing or wrong, test will fail at runtime; if SENSEVOICE_MODEL_PATH does not exist, ASR will fail

phase 4: direct-run validation (foreground test)

13. explain direct-run validation. tell user this tests voice control in foreground before enabling background mode. two processes must run simultaneously in separate terminals.

    input: user acknowledgment

    output: proceed to step 14 or stop

14. start voice service in terminal 1:

    python -m openclaw_voice_control --config config/default.yaml --env-file .env
    

    input: active venv, .env populated, OpenClaw service running

    output: voice service starts listening for wakeword, logs to stdout

    edge case: if OpenClaw is unreachable, service will error; if SENSEVOICE_MODEL_PATH is wrong, ASR will fail to load

15. start overlay UI in terminal 2 (same workspace, same time as step 14):

    python -m openclaw_voice_control.overlay_app --config config/default.yaml --env-file .env
    

    input: active venv, same .env and config/, both terminals in the same workspace

    output: overlay UI appears on screen, logs to stdout

    edge case: if both processes try to bind to the same port, second will fail; if microphone permission is denied by macOS, ASR will not record

16. test voice interaction. user speaks the wakeword (default: "hey jarvis") and gives a command. verify:

    • wakeword is recognized
    • microphone capture works
    • text is transcribed
    • response is forwarded to OpenClaw agent
    • TTS reply is played
    • overlay UI shows interaction

    input: microphone, audio output, wakeword utterance

    output: voice interaction completes, reply is heard and displayed

17. stop direct-run services before proceeding. kill both terminal processes (ctrl-c).

    input: user kills both processes or you kill them

    output: both services stopped, no orphaned processes

    edge case: if user does not stop these before starting background mode, duplicate voice runtimes will respond to same wakeword

phase 5: background deployment decision

18. ask whether user wants background resident behavior and auto-start. explain:

    • foreground validation (steps 14-17) is complete and working
    • background mode uses launchd to auto-start after login
    • background mode runs even when no terminal is open
    • user can disable or uninstall later with provided scripts

    input: user chooses "yes" or "no"

    output: proceed to step 19 (yes) or step 20 (no)

phase 6a: background deployment (if user chooses yes)

19. deploy background runtime. ask for confirmation before running ./scripts/deploy_macos.sh (this modifies launchd and creates host apps).

    input: user confirmation

    output: proceed to step 20a or stop

20a. run deployment script: bash ./scripts/deploy_macos.sh

input: workspace contains `scripts/deploy_macos.sh`, sudo may be needed

output: launchd plist files installed, host shell apps created, background service now auto-starts

edge case: if sudo is required but user does not have admin, deployment will fail; if launchd is broken, service will not start

21a. validate background service is running: bash ./scripts/doctor.sh

input: workspace contains `scripts/doctor.sh`

output: diagnostic output shows whether launchd service is active, voice process is running, env vars are correct

edge case: if service failed to start, `doctor.sh` will indicate which env var or file is missing

22a. test voice control in background mode. user speaks wakeword without any terminal running. verify interaction works.

input: wakeword utterance

output: voice control responds without manual process start

23a. explain daily maintenance. provide user with these scripts: - ./scripts/restart_service.sh to restart the background service - ./scripts/uninstall_macos.sh to remove background service and auto-start - ./scripts/doctor.sh to diagnose issues

input: none (informational)

output: user knows how to manage background service

phase 6b: stop after foreground validation (if user chooses no)

20b. end setup. voice control is ready to use in foreground mode only.

input: none

output: user can start voice service and overlay UI manually whenever needed (repeat steps 14-16)

decision points

if direct-run services are already running from a previous attempt:

ask user whether to kill old processes first. if yes, kill them. if no, stop and tell user to stop old processes manually before retrying.

if OpenClaw service is unreachable:

voice control will fail at runtime when trying to forward recognized text. stop and ask user to verify OpenClaw service is running and OPENCLAW_BASE_URL is correct.

if OPENCLAW_TOKEN is missing or invalid:

authentication will fail at first interaction. stop and ask user to retrieve the correct token from ~/.openclaw/openclaw.json (gateway key, not identity key).

if SenseVoice model path does not exist:

ASR will fail to load. stop and ask user to either download the model (step 8) or provide the correct path to an existing model directory.

if user wants to switch from openWakeWord (default) to Porcupine wakeword:

only ask for PICOVOICE_ACCESS_KEY and WAKEWORD_FILE if user explicitly requests this. do not offer Porcupine as a default option. set WAKEWORD_PROVIDER=porcupine only when user chooses it.

if user wants to change the wakeword phrase (staying with openWakeWord):

only change OPENWAKEWORD_MODEL_NAME in .env. do not download extra models unless user explicitly asks. common examples: hey jarvis (default), hey mycroft, hey rhasspy, alexa. warn that only hey jarvis has been smoke-tested.

if user says "turn it off" or "stop voice" (ambiguous intent):

ask one short clarification: "do you want to temporarily stop voice (and restart later), or completely remove it?" if temporarily stop, kill the direct-run process or run ./scripts/uninstall_macos.sh then ask if they want to reinstall. if completely remove, delete the skill folder.

if user wants to use a custom TTS voice on macOS:

that voice must be pre-downloaded in System Settings > Accessibility > Spoken Content > the i button > Language > Voice. if not, voice will default to system voice or fail. ask user to download the desired voice first, then test in direct-run mode.

if a direct-run test completed successfully but background service fails to start:

run ./scripts/doctor.sh to diagnose. common causes: launchd plist syntax error (check launchagents/), missing env var in .env or in the plist, python path mismatch (check VOICE_CONTROL_PYTHON_BIN if needed).

if network connection is offline during model download:

model download will fail and stall. inform user that step 8 or 10 failed and ask them to retry when online.

output contract

successful setup produces the following artifacts in the installed skill workspace:

• .venv/ directory with active python environment
• models/SenseVoiceSmall/ directory containing ASR model weights
• .env file in workspace root containing all 4 required values and any optional overrides
• config/default.yaml unchanged from repository (user configures via .env)
• optional: launchd plist files in ~/Library/LaunchAgents/ if background mode was enabled (step 19)
• optional: shell host app wrappers in ~/.openclaw_voice/ if background mode was enabled

direct-run processes (steps 14-15):

• voice service listens on configured port, logs startup messages
• overlay UI window appears on screen with interaction status

background service (step 20a):

• launchd service installed and active
• service auto-starts after user login
• ./scripts/doctor.sh reports healthy status
• ./scripts/restart_service.sh can restart service without manual launchd management
• ./scripts/uninstall_macos.sh can fully remove service and host apps

outcome signal

successful setup is confirmed when all of the following are true:

1. repository is synced into workspace (files present, git remote points to correct GitHub repo)
2. virtual environment is active and all dependencies installed (pip list shows openclaw-voice-control)
3. .env file exists with all 4 required values populated (test by reading back, do not print secrets)
4. direct-run test passes (step 16): user says wakeword, service recognizes it, overlay shows transcription, OpenClaw agent responds, TTS plays reply
5. if background mode enabled (step 19): ./scripts/doctor.sh reports service healthy and voice works without manual terminal process start

user will say something like: "okay, voice is working" or "i can talk to it and it understands" or "the reply came back" to confirm outcome signal.

if any artifact is missing, test fails, or service does not auto-start, offer troubleshooting: run ./scripts/doctor.sh, check .env for missing values, verify OpenClaw service is re