voice-agents

SKILL.md

Natural conversation with AI through speech, balancing latency against control.

Choose between speech-to-speech models (lowest latency, less controllable) or pipeline architectures (STT→LLM→TTS for fine-grained control)

Core challenges: latency budgeting across all components, voice activity detection, barge-in handling, and turn-taking to avoid awkward pauses or overlaps

Requires semantic VAD, response length constraints in prompts, and noise handling to achieve natural conversational flow

Works alongside agent orchestration, tool builders, and LLM architects for multi-modal agent systems

Voice Agents

Voice agents represent the frontier of AI interaction - humans speaking
naturally with AI systems. The challenge isn't just speech recognition
and synthesis, it's achieving natural conversation flow with sub-800ms
latency while handling interruptions, background noise, and emotional
nuance.

This skill covers two architectures: speech-to-speech (OpenAI Realtime API,
lowest latency, most natural) and pipeline (STT→LLM→TTS, more control,
easier to debug). Key insight: latency is the constraint. Humans expect
responses in 500ms. Every millisecond matters.

84% of organizations are increasing voice AI budgets in 2025. This is the
year voice agents go mainstream.

Principles