elevenlabs-scribe

What it does

Translates audio/video into text with timing, speaker labeling, and metadata suitable for podcasts, interviews, captions, localization prep, and QA. It explicitly notes it is not for text-to-speech, dubbing, voice cloning, or translation.

How it works

• Uses ElevenLabs Scribe models (scribe_v2 for batch transcription; scribe_v2_realtime for live streaming) and deprecates scribe_v1. Treats v1 as retired/unsafe for new work.
• Batch endpoint: POST https://api.elevenlabs.io/v1/speech-to-text with multipart form data; provide exactly one file input path or URL source; supports various audio/video formats; apply conservative preflight limits (3 GB file size and 1 hour multichannel duration) when conflicting docs exist.
• Core response includes language_code, language_probability, text, and words with optional fields like start/end, type, speaker_id, characters, logprob.
• Configurable parameters include model_id="scribe_v2", timestamps_granularity (word by default; character or none as options), tag_audio_events, diarize, num_speakers, diarization_threshold, keyterms, no_verbatim, temperature, seed.
• Supports two main modes: multichannel (use_multi_channel, fixed up to 5 channels with separate transcripts) and diarization (diarize=true with optional num_speakers; speaker_0 labels are cluster IDs, not identity-confirmed persons).
• Advanced features cover entity detection/redaction with categories like pii/phi/pci/etc., webhook-based async jobs, and captions/subtitles via formats such as SRT/VTT derived from word timestamps.
• Captions and edit prep emphasize preserving word-level timestamps, converting to turn-level speaker segments, and keeping raw transcripts as source of truth; treat SRT/VTT as derived artifacts.
• Async workflows can use webhooks with webhook=true, delivery via HTTPS, and custody metadata for provenance; webhook payloads may contain transcript text.
• Privacy, retention, and consent rules require user rights and consent verification, with options like Zero Retention Mode for certain enterprise setups; data handling varies by environment and policy.

When to use it

• Use for transcripts, caption timing, interview logs, podcast edit prep, accessibility captions, localization prep, call/interview analysis, and transcript cleanup. Do not use for TTS, dubbing, voice cloning, or translation unless the transcript is verified and rights exist.
• Choose batch vs. real-time based on whether the task is post-production (batch) or live/interactive monitoring (realtime).
• Enable diarization when speaker turns matter and channels are not isolated; use multichannel mode when each participant is isolated on a separate channel.
• Enable entity detection/redaction when sensitive data handling is needed; apply redaction as a first-pass before human review.

What it can touch

• API endpoint: POST https://api.elevenlabs.io/v1/speech-to-text
• Model IDs: "scribe_v2", "scribe_v2_realtime"
• Parameters: language_code, timestamps_granularity, tag_audio_events, diarize, num_speakers, diarization_threshold, keyterms, no_verbatim, temperature, seed, use_multi_channel, multichannel_output_style, detect_speaker_roles, use_speaker_library, webhooks, webhook_id, webhook_metadata
• Outputs include transcripts, word objects with start/end, and per-word speaker_id; caption exports in variants like SRT/VTT

Caveats

• Scribe v1 deprecated; prefer v2 and migrate if still using v1.
• Documentation contains conflicting limits on file size (3 GB conservative vs 5.0 GB) and multichannel duration (1 hour vs 10 hours); use the conservative figures (3 GB, 1 hour) for planning unless confirmed otherwise.
• Pricing and surcharges exist for entity detection, redaction, keyterm prompting, and role detection; quote conservatively and verify current numbers before quoting a client.
• Privacy and retention policies include Zero Retention Mode for some enterprises; live/transcript storage restrictions may apply, with sensitive webhook payloads.
• Diarization labels (speaker_0, etc.) are not identity-verified persons; human-map naming is required.