XiaoZhi AI Device (ESP32) integration for OpenClaw. Enables real-time voice interaction with your AI assistant through XiaoZhi hardware. Supports WebSocket b...
---
name: xiaozhiclaw
description: XiaoZhi AI Device (ESP32) integration for OpenClaw. Enables real-time voice interaction with your AI assistant through XiaoZhi hardware. Supports WebSocket bridge, Volcengine Doubao STT/TTS, and Opus audio streaming.
---
# XiaoZhiClaw - XiaoZhi AI Device Integration
## ๐ Security
- โ
No external API keys stored in code
- โ
All credentials via environment variables
- โ
No shell command execution
- โ
WebSocket connections only (no inbound HTTP)
- โ
Open source and auditable
- โ ๏ธ Requires Volcengine Doubao API credentials
## Overview
XiaoZhiClaw is an OpenClaw channel that connects XiaoZhi AI ESP32 hardware devices to OpenClaw agents, enabling real-time voice interaction.
## Permissions
### Required Permissions
- โ
Network Access: WebSocket server (port 8080 by default)
- โ
Audio Processing: Opus encoding/decoding
- โ
STT/TTS API: Volcengine Doubao (HTTPS)
- โ No Admin/Root Privileges Required
- โ No System Command Execution
### Data Flow
```
XiaoZhi Device โ WebSocket โ STT (Doubao) โ OpenClaw Agent
โ โ
Microphone AI Response
โ โ
Speaker โ WebSocket โ TTS (Doubao) โ OpenClaw Agent
```
## Use Cases
### 1. Voice Conversation
```
Talk to your AI assistant through XiaoZhi hardware
Ask questions and get voice responses
Real-time voice interaction
```
### 2. Hardware Control
```
Control volume, brightness via MCP commands
Hardware status monitoring
Device management
```
### 3. Voice Commands
```
Voice-activated AI assistant
Hands-free operation
Physical AI companion
```
## Usage Examples
### Start the Service
```bash
# The WebSocket server starts automatically when OpenClaw starts
# Default port: 8080
```
### Configure XiaoZhi Device
Configure your XiaoZhi firmware to connect to:
```
ws://YOUR_COMPUTER_IP:8080
```
### Voice Interaction Flow
1. **User speaks** โ XiaoZhi microphone captures audio
2. **Audio streaming** โ Opus frames sent via WebSocket
3. **STT processing** โ Volcengine Doubao transcribes to text
4. **AI processing** โ OpenClaw agent processes and responds
5. **TTS processing** โ Volcengine Doubao converts to speech
6. **Audio playback** โ XiaoZhi speaker plays response
## Environment Variables
```bash
# Required: Volcengine Doubao API Credentials
# Get from: https://console.volcengine.com/
DOUBAO_APP_ID=your_app_id_here
DOUBAO_ACCESS_TOKEN=your_access_token_here
# Optional: WebSocket Server Configuration
XIAOZHI_PORT=8080
# Optional: Audio Configuration
AUDIO_SAMPLE_RATE=16000
AUDIO_FRAME_DURATION=60
```
## Protocol
### WebSocket Message Types
**Handshake:**
```json
{
"type": "hello",
"transport": "websocket",
"audio_params": {
"format": "opus",
"sample_rate": 16000,
"frame_duration": 60
}
}
```
**Listen Events:**
```json
{
"type": "listen",
"state": "start"
}
```
```json
{
"type": "listen",
"state": "stop",
"text": "transcribed text"
}
```
**TTS Events:**
```json
{
"type": "tts",
"state": "start",
"text": "response text"
}
```
```json
{
"type": "tts",
"state": "stop"
}
```
## Architecture
```
XiaoZhi ESP32 โโ WebSocket Server โโ OpenClaw Channel โโ AI Agent
โ โ โ โ
Microphone Port 8080 xiaozhiclaw PocketAI
โ โ โ โ
Speaker Opus Audio Message Router Response
โ
Doubao STT/TTS
```
## Notes
1. **Network:** Ensure port 8080 is open on your firewall
2. **Latency:** Use wired connection or high-speed Wi-Fi for best results
3. **API Credentials:** Volcengine Doubao API credentials required for STT/TTS
4. **Audio Format:** Opus encoding, 16kHz sample rate, 60ms frame duration
## Troubleshooting
### Connection Refused
- Check if port 8080 is open
- Verify XiaoZhi device network settings
- Check firewall settings
### Audio Lag
- Check network latency
- Use wired connection if possible
- Ensure good Wi-Fi signal strength
### STT/TTS Not Working
- Verify Volcengine API credentials
- Check API quota and billing
- Verify network connectivity to Volcengine API
### Device Not Connecting
- Verify WebSocket URL format: `ws://IP:PORT`
- Check XiaoZhi firmware configuration
- Ensure OpenClaw gateway is running
## Resources
- [XiaoZhi AI ESP32 Project](https://github.com/xiaozhi-ai)
- [Volcengine Doubao API](https://www.volcengine.com/)
- [OpenClaw Documentation](https://docs.openclaw.ai/)
- [Opus Audio Codec](https://opus-codec.org/)
## Changelog
### v1.0.0 (2026-03-12)
- โ
Initial release
- โ
WebSocket server implementation
- โ
Volcengine Doubao STT/TTS integration
- โ
Opus audio encoding/decoding
- โ
Real-time voice conversation
- โ
OpenClaw channel integration
## License
MIT License
## Author
PocketAI for Leo - OpenClaw Community
## Credits
- OpenClaw Team
- XiaoZhi AI ESP32 Project
- Volcengine Doubao
- PocketAI ๐งค
don't have the plugin yet? install it then click "run inline in claude" again.
by @clawhub