Skip to main content
Mode: "speed" (default) Best for: fast turnaround, batch jobs, and workflows where time matters more than perfect lip-sync. For higher fidelity at the cost of latency, see Precision mode. For dozens or hundreds of videos in one job, see Video Translation batches.

How Lip Sync and Video Translation Relate

Both APIs sit on top of one lip sync engine that runs in two modes, Speed and Precision, trading latency for fidelity. What differs is what each API does around that engine:
  • Lip Sync API is the engine on its own. You bring the video and your own audio; the engine redraws the mouth to match. Dialogue replacement only, no translation or audio generation.
  • Video Translation API transcribes, translates and generates the translated audio, then optionally runs the engine:
    • Audio only (translate_audio_only: true) stops after the audio. The output is the new audio track (or the original video with swapped audio and an untouched mouth).
    • Video (audio + visual) runs the lip sync engine so the mouth matches the translated speech.

Quick Start

1. List Supported Languages

Fetch the available target language codes via GET /v3/video-translations/languages:

2. Submit a Translation

Full schema: POST /v3/video-translations. List several output_languages to translate into all of them in one request; the response returns one ID per language in video_translation_ids.
The source video is either { "type": "url", "url": "..." } or { "type": "asset_id", "asset_id": "<asset_id>" }. The URL must be publicly accessible (test by opening it in an incognito browser). To use an asset_id, upload first via POST /v3/assets, as the Upload Assets guide shows.

3. Poll for Status

Poll GET /v3/video-translations/{video_translation_id} while status is pending or running. On completed, video_url is available; on failed, read failure_message. Skip polling by passing callback_url (and your own callback_id, echoed back) as described in Webhooks.

Speed Mode Options

Every optional parameter (speaker_num, translate_audio_only, disable_music_track, enable_speech_enhancement and more) is in the create reference. Two behave in ways worth knowing:
  • enable_dynamic_duration (default true) lets the output duration vary to match natural speech pacing. Billing is unaffected: translations are always charged on the source duration.
  • brand_glossary_id applies a brand glossary (see Keep terminology intact). brand_voice_id is its deprecated former name; both are accepted and resolve to the same glossary.

Keep terminology intact

A brand glossary tells the translator which terms to leave alone and which to replace with a fixed form. Create one with POST /v3/brand-glossaries:
Glossary create payload
Then pass the returned brand_glossary_id at the top level of the translation request:
Translation request
Its do_not_translate_terms stay untranslated and its forced_translations get their exact replacement. Rules apply case-insensitively in every target language of the request. A translation is inserted verbatim, so pick a form that reads correctly without inflection. Glossaries created in the HeyGen app under Brand Kit carry the same rules and work the same way.

Stock Voice (Enterprise)

By default, Video Translation clones the original speaker, so the translated video sounds like them. A preset “stock” voice instead speaks the translation in a natural voice optimized for clear pronunciation and accent in the target language. The trade-off: the result won’t sound like the original speaker.
This is an Enterprise feature, available for selected accounts and languages and turned on by request. To use it, contact your HeyGen account team.
Add stock_voice_config to the translation request. preferred_stock_voice_ids is optional and pins specific voices; each entry is a Gemini voice name (e.g. Erinome, Vindemiatrix), not a UUID. Without it, the target language’s default stock-voice pool is used.

Captions

Captions are generated in SRT and VTT. A completed GET /v3/video-translations/{video_translation_id} carries them as presigned download URLs in srt_caption_url and vtt_caption_url, alongside video_url. Each field appears once status is completed and that caption file is available. A field whose URL has not been produced is left out of the response entirely rather than returned as null, so read both with a presence check before using them. The enable_caption request field is retained for compatibility and no longer changes anything, since whether captions are displayed is a download-side choice.

Proofread Before Finalizing

Both modes support the proofread workflow: review and edit subtitles before spending credits on final generation.
  1. Create a session with POST /v3/video-translations/proofreads. It takes the same video, output_languages, title and mode as a translation and returns proofread_ids, one per language.
  2. Poll GET /v3/video-translations/proofreads/{proofread_id} until completed.
  3. Download and edit the SRT. GET /v3/video-translations/proofreads/{proofread_id}/srt returns an srt_url. Edit the file locally, then upload the revised version with Upload Proofread SRT:
  4. Generate the final video with POST /v3/video-translations/proofreads/{proofread_id}/generate, body { "captions": true }. It returns a video_translation_id to poll as in step 3 of the Quick Start.
GET /v3/video-translations/proofreads lists sessions and DELETE /v3/video-translations/proofreads/{proofread_id} removes one; see Precision mode.

Other Operations

GET /v3/video-translations lists translations (e.g. ?limit=10), paged with has_more and next_token. DELETE /v3/video-translations/{video_translation_id} removes one.

When to Use Speed vs. Precision

For high-volume jobs across many source videos, see Video Translation batches.