fish-audio-tts

What it does

The skill enables text-to-speech generation and voice cloning using Fish Audio. It supports hosted API models (S2.1-Pro, S2-Pro, S1) with REST and WebSocket streaming, and open-weight OpenAudio models for self-hosting. It covers cloning via persistent models or instant cloning from reference audio, supports inline emotion/tone/paralinguistic markers, cost estimation based on UTF-8 bytes, and streaming for real-time voice agents. It also helps decide between self-hosting or using the hosted API and to review output for production.

How it works

• Uses Fish Audio hosted API endpoints for TTS and cloning, including REST: POST https://api.fish.audio/v1/tts and WebSocket: wss://api.fish.audio/v1/tts/live, plus POST https://api.fish.audio/model for cloning.
• Sets required headers: Authorization: Bearer <FISH_API_KEY>, model: <model-name>, and Content-Type accordingly.
• For TTS, sends parameters such as text, reference_id, references, format, mp3_bitrate, sample_rate, speed, latency, chunk_length, normalize, temperature, top_p, repetition_penalty, max_new_tokens.
• Supports streaming via WebSocket with events: start, text, flush, stop from client; audio and finish from server.
• Uses inline markers in text for emotion/tone: S1 uses parentheses like (excited); S2 uses square brackets like [excited]. Marker syntax must match the chosen model and is described in the skill.
• For cloning, uses either persistent models with POST /model or instant cloning via references in a TTS call. Reference audio should be 30–60 seconds (ideal), mono, single speaker, and provided in formats like .wav, .mp3, .m4a, .opus.
• Includes OpenAudio open weights for self-hosting, notably OpenAudio S1-mini, with licensing and hosting considerations; notes that OpenAudio S1-mini is CC-BY-NC-SA-4.0 and not suitable for commercial use without a separate license.

When to use it

• When narration or dialogue must be produced with expressive voice cloning and real-time streaming.
• When deciding between hosted API usage versus self-hosting with open weights.
• When needing to estimate cost based on UTF-8 bytes and to review TTS output for production suitability.

What it can touch

• External endpoints: REST and WebSocket endpoints for Fish Audio hosted API.
• Models via POST /model for cloning, and via model header for TTS calls.
• Media inputs/outputs in formats such as mp3, wav, pcm, opus; streaming as audio chunks.

Caveats

• Hosted API and open weights are not the same models and have different licenses; OpenAudio S1-mini weights are CC-BY-NC-SA-4.0 (non-commercial). Commercial self-hosting requires a separate license or use of the hosted API. Verification date for dated claims is 2026-07-10.
• Open weights require careful licensing checks before commercial use; the weights are non-commercial licensed, while hosted API is commercial and priced by UTF-8 bytes.
• Marker syntax compatibility depends on model: S1 uses parentheses, S2 uses square brackets; ensure the model header matches the marker syntax.