This is a tutorial on using the AI intelligent translation and editing software created by GhostCut. In this document, it explains in detail how the AI extracts speech, translates subtitles, and aligns subtitles, audio, and images. Compared to other translation products such as Veed, Capcut+Deepl, GhostCut auto video translation is simpler and more automated. It is an essential translation software for bloggers on platforms like TikTok and Youtube.

Principles of Video Translation and AI Dubbing

Let's take a look at a real case of using GhostCut to translate and voiceover a YouTube video：

Original video - Russian voice-over

original.mp4

New video - Chinese language with background music reserved

Chinsese.mp4

This function uses ASR technology to recognize the "voice" in the original video and generate translated subtitles.

The principle of translating video audio usually involves the following steps:

  1. Separation of audio, image, and subtitles: Separate the audio, image, and subtitle files from the video and split them into different files.
  2. [ASR] Speech recognition of audio: Use speech recognition technology to convert speech in the audio file into a text file.
  3. Machine translation of text: Use machine translation technology to convert the original text in the text file to text in the target language.
  4. Speech synthesis of the target language text: Use speech synthesis technology to convert the text in the target language to speech in the target language.
  5. Synchronize the target language speech with the original video: Use audio alignment technology to synchronize the speech in the target language with the original video to form a translated video.

GhostCut supports online subtitle editing and downloading of SRT subtitles, and supports multiple font styles. Compared to using multiple editing and translation software, the feature of GhostCut for translating video and voiceover is: fully automatic, with one-click deduplication and removal of original subtitles, and automatic audio and video alignment.

Note: If the original video only has subtitles without voiceover, please use the Auto Hardcode Subtitles Translation function, click here to view the tutorial for translating video building text.

Auto Hard Sub Translation-Guides-GhostCut

😀-Translation Example with Good Quality

GhostCut uses lots of AI tech for video translation, including speech separation,ASR, subtitle translation, error correction, text simplification, alignment, and more. The following are areas that can easily affect accuracy: