elevenlabs-dubbing-voice-conversion

What it does

The skill directs an agent to work on transforming existing speech for ElevenLabs workflows, covering dubbing/localization, voice-changing/speech-to-speech, and voice isolation/enhancement. It distinguishes when to choose each ElevenLabs path and when to avoid using it for plain text-to-speech. It also identifies steps to plan, document, and export dubs and transcripts, while considering consent, rights, and cost considerations.

How it works

• Start by choosing the ElevenLabs path before touching assets:
  • Dubbing/localization: select Dubbing when translating audio/video to another language while preserving timing and speaker identity as much as possible; plan transcript review, language QA, and media export.
  • Voice changer / speech-to-speech: select Voice Changer API when the language/content is correct and the goal is to transform an existing performance into another voice while maintaining emotion, rhythm, timing, and delivery.
  • Voice isolation: run Audio Isolation before dubbing/conversion when noise or background elements hinder recognition; do not apply blindly to clean audio.
  • External localization: plan for human translation/review if exact legal/medical/brand translation and strict lip-sync are required.
  • Generic TTS: use a separate skill for scripted voiceover; this one covers cases where existing speech or timing is central.
• Follow documented API details for Dubbing, Voice Changer, and Audio Isolation, including how to format requests, parameters like source_lang, target_lang, num_speakers, watermark, enable_logging, model_id, and file handling.
• Prepare source media with rights, consent, format, timing, speaker details, language, background audio decisions, terminology, and pilot spans.
• Execute the appropriate workflow sequence for Dubbing, Voice Changer, or Audio Isolation, including pilot runs, job creation, status polling, transcript and media exports, review, and potential repairs.
• Consider pricing and privacy constraints, particularly zero-retention options and consent requirements for voice cloning.

When to use it

• Use when transforming existing speech into another language or voice while preserving timing and speaker identity as feasible.
• Use when considering voice cloning or speech-to-speech transformations and when background noise needs cleaning prior to conversion.
• Use when planning localization deliverables and ensuring proper rights, consent, and export of dubs/transcripts.

What it can touch

• Interfaces with ElevenLabs Dubbing API endpoints (e.g., create, get, audio/get, transcripts/get).
• Voice Changer API at POST /v1/speech-to-speech/{voice_id} with options like output_format, model_id, voice_settings, seed, remove_background_noise, file_format, and enable_logging.
• Audio Isolation at POST /v1/audio-isolation with multipart audio and specific PCM constraints for low latency.

Caveats

• Pricing is volatile and must be re-checked before quoting budgets.
• Dubbing v2 API behavior is not guaranteed as live; do not promise v2 capabilities unless current docs confirm changes.
• Zero Retention Mode exists for select Enterprise customers; not all dubbing inputs/outputs are eligible for retention.
• Voice cloning requires consent verification; responsibility for authorized use rests with the creator, and cloning reliability varies with overlap and voice similarity.
• Automatic dubbing is a localization tool with risks in names, terms, idioms, and lip-sync; a pilot and review step are recommended.