voice recognition , smart auto-model selection

transcribe audio to text using local openai whisper. no api keys, no internet required, 100% private.

smart auto-selection dynamically picks the best model based on your audio characteristics. you never have to think about which model to use.

intent

use this skill to transcribe audio files into text with zero external dependencies, zero cost, and zero privacy risk. the skill analyzes audio duration and complexity, then automatically selects the optimal whisper model (base, small, or medium) to balance speed and accuracy. run it when you need to convert voice messages, meeting recordings, interviews, podcasts, or any speech input in 99+ languages into written text. best for workflows where you control the audio file locally and want full privacy with no api quotas or rate limits.

inputs

required

• audio file path (string): local path to audio file in one of these formats: ogg, wav, mp3, m4a, flac, opus. file must be readable and under 500mb (whisper processes streaming, so no hard limit, but very large files take proportionally longer).
• python 3.10 or higher: runtime environment.
• openai-whisper package: install via pip install openai-whisper soundfile numpy or use bundled installer.
• pytorch: install cpu version via pip install torch --index-url https://download.pytorch.org/whl/cpu (gpu optional but not required).

optional

• language code (string): iso 639-1 code (e.g., "en", "zh", "yue", "es"). if omitted, whisper auto-detects. provide this as a hint if auto-detection fails or for faster inference.
• model override (string): force a specific model ("tiny", "base", "small", "medium", "large"). if omitted, auto-selection runs.
• output file path (string): if specified, transcript writes to this file instead of stdout.
• segment timestamps flag (boolean): if true, include sentence-level timing data.

external connections

• none required. all processing is local. first run downloads the selected whisper model (~139mb for base, ~461mb for small, ~1.5gb for medium) from openai's huggingface repo to ~/.cache/whisper/. subsequent runs use cached model.

procedure

1. load and validate audio file (input: audio file path; output: audio object, duration in seconds, sample rate in hz)

   • read file from disk using soundfile library.
   • if file is missing or unreadable, exit with error "file not found" or "file is not a valid audio format".
   • extract duration and sample rate metadata.

2. analyze audio characteristics (input: duration, sample rate; output: complexity score, recommended model)

   • measure duration in seconds.
   • scan audio for silence, speech gaps, and language transitions to estimate complexity (low = clean monolingual, high = mixed languages, background noise, accents).
   • assign complexity flag: 0 (clean), 1 (mixed/noisy).

3. select model (input: duration, complexity flag, optional model override; output: model name)

   • if user provided --model flag, skip to step 4 using that model.
   • else apply auto-selection rule (see decision points below).

4. download or load model (input: model name; output: loaded whisper model, load time in seconds)

   • check if model exists in ~/.cache/whisper/.
   • if not cached, download from huggingface (first run only, ~10-30s depending on model size and network speed).
   • if download fails (network timeout, corrupted file), retry up to 2 times with exponential backoff (2s, 4s), then exit with error.
   • if cached, load from disk (~1-3s).
   • on timeout or network error after retries, output "model download failed: check internet connection or cache manually".

5. transcribe audio (input: loaded model, audio object, language hint (optional); output: transcript string, confidence scores per segment)

   • pass audio and language hint (if provided) to whisper.
   • whisper processes audio in 30-second chunks internally.
   • if language is auto-detected, whisper detects it during first chunk.
   • collect output: full transcript text + per-segment metadata (start time, end time, text, confidence).
   • if transcription fails (cuda out of memory, corrupted audio), fall back to cpu inference (if gpu was used) or exit with "transcription failed: audio may be corrupted".

6. format and output transcript (input: transcript text, segment metadata, output file path (optional), segment flag (optional); output: formatted text or file)

   • if --segments flag is set, format output as:

     [full transcript here]
     
     📝 Segments:
        [0.0s - 3.6s] segment text here
        [3.6s - 7.4s] next segment text
     

   • else output full transcript only.
   • if output file path is specified, write result to file (create or overwrite). if file write fails (permission denied, disk full), exit with error.
   • if no output file specified, print to stdout.

7. log performance metrics (input: model name, total duration, transcription time; output: console output with timing)

   • display: audio duration, sample rate, selected model, model load time, transcription time, total elapsed time.

decision points

if user provides explicit model flag (--model)

• skip auto-selection. use the specified model. proceed to step 4.
• else, continue to next decision.

if audio duration < 10 seconds

• if complexity is low (clean speech, single language), select base model.
• else (mixed language, background noise, accents detected), select small model.
• proceed to step 4.

if audio duration 10-60 seconds

• if complexity is low, select base model.
• else, select small model.
• proceed to step 4.

if audio duration 1-2 minutes

• select small model (balances accuracy and speed).
• proceed to step 4.

if audio duration > 2 minutes

• select medium model (maximum accuracy for long recordings).
• proceed to step 4.

if model download fails due to network error

• retry up to 2 times with exponential backoff (2s, then 4s).
• if all retries fail, exit with error "unable to download model. check internet connection. you can manually download from huggingface and place in ~/.cache/whisper/".
• do not fall back to a smaller model unless user explicitly requests it.

if language is not specified

• whisper auto-detects language during transcription (step 5).
• if confidence in auto-detected language is very low (< 0.5), output warning to stderr: "language detection confidence is low. consider providing --language hint".
• do not halt transcription. continue with auto-detected language.

if output file path is specified but file cannot be written

• exit with error: "cannot write to [filepath]: permission denied or disk full".
• do not fall back to stdout. user explicitly requested file output.

if audio is very short (< 1 second)

• still process normally with base model.
• output warning: "audio is very short (< 1s). transcript may be incomplete or empty".

output contract

on success

• transcript: plain text string containing the full transcription of the audio.
• location: stdout (console) or specified output file (if -o flag used).
• format:
  • basic: single block of text with full transcript.
  • with segments: full transcript followed by "📝 Segments:" header and list of [start_time - end_time] segment_text, each on one line.
• metadata logged to stderr: duration, sample rate, selected model, load time, transcription time, total time.
• exit code: 0 on success.

on failure

• exit code: 1.
• error message to stderr describing the failure (file not found, model download failed, transcription failed, etc.).
• no partial output written to file. file either contains full valid transcript or is not created/modified.

outcome signal

you know the skill worked when:

1. the console displays a summary line like "✅ done (4.1s total)" or similar completion message.
2. the transcript text appears on stdout (or in the specified output file if -o was used).
3. if you used --segments, you see timestamped segments with start and end times.
4. exit code is 0 (check with echo $? on unix/linux).
5. no errors printed to stderr (other than optional warnings like low language confidence).
6. the transcribed text matches the actual speech content of the audio (spot check: listen to a 5-10s clip and verify the transcript is accurate).