openai-audio

What it does

The skill enables an AI agent to produce and understand audio using OpenAI request-based audio APIs and audio-capable chat models. It covers text-to-speech, transcription, translation, multimodal audio input/output, model routing, prompt and performance direction, artifact custody, approval gates, and safety/rights review. It is intended for non-realtime, bounded audio production, with live voice routing handled by a separate realtime skill.

How it works

The skill provides a structured framework for selecting endpoints and models based on user intent (e.g., generate narration via /v1/audio/speech, transcribe via /v1/audio/transcriptions, translate via /v1/audio/translations, or add audio to chat via gpt-audio-1.5). It documents model options (e.g., gpt-4o-mini-tts, tts-1/tts-1-hd, gpt-4o-transcribe, whisper-1), output formats (wav, mp3, aac, etc.), and streaming vs. non-streaming behavior. It outlines a decision table that pairs user intent with primary route, model/API choice, and notes about performance, latency, and quality. It emphasizes separating request-based tasks from realtime routing and requires explicit approval gates for public-facing voices, including rights checks and audition approvals. It also prescribes lifecycle custody for artifacts (masters and delivery copies) and provides example code snippets illustrating how to invoke speech endpoints with specific models, voices, scripts, and instructions. The procedure includes handling of licensing, consent, and disclosing AI-generated voices per policy.

When to use it

Use this skill for bounded, non-realtime audio tasks such as generated narration, podcast transcription, subtitle timing, translation to English, audio QA, or integrating audio input/output into an existing chat workflow. Do not rely on it for continuous low-latency speech-to-speech, live interpreting, SIP/WebRTC sessions, or voice activity detection; those are handled by the separate realtime voice skill. If a route is blocked, you should report the failure (model access, policy, privacy, cost, or product mismatch) and ask before switching endpoints or models.

What it can touch

The skill references and relies on OpenAI API endpoints and models, including "/v1/audio/speech", "/v1/audio/transcriptions", "/v1/audio/translations", and the audio-enabled chat pathway with models like gpt-audio-1.5. It discusses output formats (wav, mp3, aac, opus, flac, pcm), voice options (e.g., built-in voices like marin and cedar), and settings such as speed and instructions length. It also mandates keeping audits of artifacts, such as source script, model, voice, instructions, response format, speed, timestamps, approvals, and checksums.

Caveats

The skill notes that pricing for realtime and transcription is volatile and should be rechecked before quoting costs in production. It also references data-control policies: certain endpoints have different retention for abuse-monitoring, and some data handling policies require disclosure and consent for AI-generated voices. It emphasizes compliance with usage policies regarding likeness and consent for voice work, and that custom voices require separate consent and recording workflows. It cautions to route to the realtime skill for live sessions and to follow approval gates for public-facing voices.