Configure and use Gladia audio intelligence features: speaker diarization, translation, sentiment analysis, named entity recognition (NER), PII redaction, su...
SKILL.md

---
name: audio-intelligence
description: "Configure and use Gladia audio intelligence features: speaker diarization, translation, sentiment analysis, named entity recognition (NER), PII redaction, subtitles (SRT/VTT), summarization, chapterization, custom vocabulary, and audio-to-LLM. Use when the user asks about any audio intelligence feature, enabling features on pre-recorded or live transcription, understanding which features are available in each mode, or combining multiple features. Always prefer the official SDK; fall back to raw REST only when SDK cannot satisfy the requirement."
license: MIT
---

# Audio Intelligence

Gladia's audio intelligence features extract structured data and insights from transcripts. They work on top of the base transcription — most are enabled by adding options to the `transcribe()` call (pre-recorded) or the `startSession()` config (live).

> **SDK-first**: always use the official SDK — see [sdk-integration](../sdk-integration/SKILL.md) for policy, setup, and fallback criteria.

## When to Use

- User asks about a specific feature: diarization, translation, PII redaction, sentiment, NER, subtitles, summarization, etc.
- Enabling or configuring one or more audio intelligence features on pre-recorded or live transcription
- Understanding which features are available in live vs pre-recorded mode
- Combining multiple features in a single transcription job

**When NOT to use:** For basic transcription without audio intelligence features, go directly to [pre-recorded-transcription](../pre-recorded-transcription/SKILL.md) or [live-transcription](../live-transcription/SKILL.md). For gotchas and errors related to specific features, see [troubleshooting](../troubleshooting/SKILL.md).

## References

Consult these resources as needed:

- ./references/live-audio-intelligence.md -- Detailed config and WebSocket responses for all live-mode features
- ./references/pre-recorded-audio-intelligence.md -- Detailed config and response structures for all pre-recorded audio intelligence features
- ../pre-recorded-transcription/SKILL.md -- Pre-recorded transcription workflow and options
- ../live-transcription/SKILL.md -- Live transcription session config and event handling
- ../sdk-integration/SKILL.md -- SDK setup, client initialization, and SDK vs raw API decision guide
- ../troubleshooting/SKILL.md -- Common errors, gotchas, and verification checklist

## Feature Availability

| Feature                  | Pre-recorded | Live | Config key                      |
| ------------------------ | :----------: | :--: | ------------------------------- |
| Speaker diarization      |     Yes      |  No  | `diarization`                   |
| Translation              |     Yes      | Yes  | `translation`                   |
| Sentiment analysis       |     Yes      | Yes  | `sentiment_analysis`            |
| Named entity recognition |     Yes      | Yes  | `named_entity_recognition`      |
| Subtitles (SRT/VTT)      |     Yes      |  No  | `subtitles`                     |
| Custom vocabulary        |     Yes      | Yes  | `custom_vocabulary`             |
| PII redaction            |     Yes      |  No  | `pii_redaction`                 |
| Chapterization           |     Yes      | Yes  | `chapterization` (post-process) |
| Summarization            |     Yes      | Yes  | `summarization` (post-process)  |
| Audio-to-LLM             |     Yes      |  No  | `audio_to_llm`                  |
| Custom spelling          |     Yes      | Yes  | `custom_spelling`               |
| Custom metadata          |     Yes      | Yes  | `custom_metadata`               |

Live features split into two groups: **real-time** (results stream during the session) and **post-processing** (results arrive after `stopRecording()`). See [./references/live-audio-intelligence.md](./references/live-audio-intelligence.md) for details.

## Quick Config Examples

Code examples assume `GladiaClient` is already initialized — see [sdk-integration](../sdk-integration/SKILL.md) for setup.

### Speaker Diarization (pre-recorded only)

```typescript
const result = await client.preRecorded().transcribe("audio.mp3", {
  diarization: true,
  diarization_config: { number_of_speakers: 2 },
});
// Each utterance includes a `speaker` field (0-indexed integer)
```

```python
result = client.prerecorded().transcribe("audio.mp3", {
    "diarization": True,
    "diarization_config": {"number_of_speakers": 2},
})
```

### Translation (pre-recorded and live)

**Pre-recorded:**

```typescript
const result = await client.preRecorded().transcribe("audio.mp3", {
  translation: true,
  translation_config: { target_languages: ["fr", "es"] },
});
```

```python
result = client.prerecorded().transcribe("audio.mp3", {
    "translation": True,
    "translation_config": {"target_languages": ["fr", "es"]},
})
```

**Live** (result streams as `translation` WebSocket events — see [live-audio-intelligence.md](./references/live-audio-intelligence.md)):

```typescript
const session = client.liveV2().startSession({
  // ... audio format options ...
  realtime_processing: {
    translation: true,
    translation_config: { target_languages: ["fr"] },
  },
});
```

```python
from gladiaio_sdk import LiveV2InitRequest, LiveV2RealtimeProcessing

session = client.live().start_session(
    LiveV2InitRequest(
        # ... audio format options ...
        realtime_processing=LiveV2RealtimeProcessing(
            translation=True,
            translation_config={"target_languages": ["fr"]},
        ),
    )
)
```

### Summarization (pre-recorded and live)

**Pre-recorded:**

```typescript
const result = await client.preRecorded().transcribe("audio.mp3", {
  summarization: true,
  summarization_config: { type: "bullet_points" },
});
```

**Live** (arrives after `stopRecording()` as `post_summarization` event):

```typescript
const session = client.liveV2().startSession({
  // ... audio format options ...
  post_processing: {
    summarization: true,
    summarization_config: { type: "bullet_points" },
  },
});
session.on("message", (msg) => {
  if (msg.type === "post_summarization") console.log(msg.data.results);
});
```

For full per-feature config options and response structures, see:

- Pre-recorded: [./references/pre-recorded-audio-intelligence.md](./references/pre-recorded-audio-intelligence.md)
- Live: [./references/live-audio-intelligence.md](./references/live-audio-intelligence.md)

## Common Mistakes

- **`code_switching: true` with empty `languages`**: triggers evaluation across 100+ languages and causes frequent misdetections. Always provide 3-5 expected languages.
- **Custom vocabulary `intensity` above 0.6**: values over 0.6 cause false positives where unrelated words get replaced. Keep at 0.4-0.6 and use `pronunciations` for better results.
- **Expecting diarization, PII redaction, subtitles, or audio-to-LLM in live mode**: these four features are pre-recorded only.
- **Enabling many features simultaneously without considering cost/latency**: each enabled feature adds processing time. Enable only what you need; combine `diarization + summarization + translation` only when all are required.

For the full gotcha list, see [troubleshooting](../troubleshooting/SKILL.md).

## Further Reading

- [Audio intelligence overview](https://docs.gladia.io/chapters/pre-recorded-stt/audio-intelligence)
- [Speaker diarization](https://docs.gladia.io/chapters/audio-intelligence/speaker-diarization)
- [Translation](https://docs.gladia.io/chapters/audio-intelligence/translation)
- [Sentiment analysis](https://docs.gladia.io/chapters/audio-intelligence/sentiment-analysis)
- [Named entity recognition](https://docs.gladia.io/chapters/audio-intelligence/named-entity-recognition)
- [PII redaction](https://docs.gladia.io/chapters/audio-intelligence/pii-redaction)
- [Custom vocabulary](https://docs.gladia.io/chapters/audio-intelligence/custom-vocabulary)
- [Summarization](https://docs.gladia.io/chapters/audio-intelligence/summarization)
- [Chapterization](https://docs.gladia.io/chapters/audio-intelligence/chapterization)
- [Audio-to-LLM](https://docs.gladia.io/chapters/audio-intelligence/audio-to-llm)
- [Live audio intelligence](https://docs.gladia.io/chapters/live-stt/audio-intelligence)
Audio Intelligence

SKILL.md

related skills
