heygen-avatar-video

What it does

Produce HeyGen avatar videos using Direct Video, Video Agent, Digital Twin, Avatar Realtime, and related avatar/voice/asset APIs. It supports provider routing, avatar and voice selection, script-to-video workflows, lip-sync from audio, backgrounds/scenes, callbacks and polling, consent and rights checks, localization handoffs, artifact custody, QA, and troubleshooting HeyGen avatar-video productions.

How it works

• Chooses the appropriate HeyGen route depending on control vs. speed: Video Agent for prompt-driven generation with less scene control; Direct Video for explicit, repeatable parameters; Cinematic Avatar for short, non-speaking cinematic shots; Avatar Realtime for live streaming in real time.
• Performs preflight: authenticates via API key, inventories looks and voices, verifies consent state for digital twins, estimates cost, and checks file accessibility.
• Handles avatar look/consent: uses avatar_id as look ID and applies consent rules for digital twins, with a multi-level consent flow if needed; maintains an audit note with IDs and consent status.
• Selects engine for Direct Video: defaults to Avatar IV unless Avatar V is supported and requested; notes limitations and permissions for motion_prompt and expressiveness.
• Processes script and direction: writes for a human presenter with short sentences, uses motion_prompt and voice_settings where supported, enables localization with appropriate lip-sync checks.
• Applies Direct Video request pattern: includes fields like avatar_id, script, voice_id, background, caption, engine type, and callback details; supports audio_url/audio_asset_id usage when narration is pre-approved.
• Manages Video Agent prompts: provides structured prompts to generate concepts before final Direct Video rendering; ensures explicit business constraints remain in final output.
• Manages Avatar Realtime pattern: supports text_stream/tts/audio modes with upstream systems for real-time synchronization and safety checks.
• Handles backgrounds, captions, and branding: uses assets API for custom backgrounds, adds captions (SRT), and can apply watermarks as needed; preserves custody with payload and IDs.
• Polls and callbacks: monitors video status (pending, processing, completed, failed); stores video_url on success and records failures with detailed messages; applies appropriate retry logic for transient issues.
• Enforces safety and policy: requires explicit rights for identifiable likenesses; logs consent status and keeps moderation and rights information in production logs.

When to use it

Use when producing HeyGen avatar videos where the deliverable involves an AI presenter, digital twin, photo avatar, studio avatar, prompt-composed avatar clip, or real-time streaming avatar. Use Video Agent for fastest exploration; Direct Video for production-grade control; Avatar Realtime for live, real-time streams.

What it can touch

• API endpoints: POST /v3/video-agents, POST /v3/videos with type fields, POST /v3/avatar-realtime.
• Look and voice inventories: GET /v3/avatars/looks, GET /v3/voices (with language, gender, type, engine filters).
• Consent management: POST /v3/avatars/{group_id}/consent when needed.
• Asset and background handling: Assets API for backgrounds; use asset_id references in requests.
• Output and custody: captures video_id, video_url, callback_id, avatar look ID, voice ID, asset IDs, engine type, and title.
• Status and polling: monitor statuses and use video_url on completion; handle errors with detailed failure fields.

Caveats

• Pricing and engine support details are volatile; verify current pricing and engine support just before paid runs.
• Digital twins require proper consent; follow HeyGen consent levels and audit rights accordingly.
• Some features like motion_prompt/expressiveness depend on avatar/engine combinations and may be restricted.
• Callback-based workflows require proper callback URLs and custody handling; ensure public reachability of assets.