Every localization professional knows that the same text can have a different length in different languages. For example, a translation between English and Italian can differ by up to 30 percent. This means that for voice-overs, the original speech and translation may fall significantly out of sync during playback. To avoid this problem, we need to synchronize the two speech streams.
This is the most important part of the Blog, how to align subtitles, voiceover and video? We use different techniques to process text, voice and video separately for synchronization.
Text processing: By reducing unnecessary pauses between words and phrases, we can achieve synchronization. If we find that the translated subtitle is too long and cannot be aligned with the video even after the voice is accelerated, we will use an AI sentence simplification algorithm to simplify the lengthy text. Here's an example:
Original sentence: Every localization professional knows that the same text can have a different length in different languages.
Simplified sentence: Localization professionals know text length varies by language.
Voice processing: To optimize the sound to match the video, we can use timings to generate audio of the desired duration. However, if the voices still don’t sync up, the algorithm will accelerate the speech rate.
Video processing: After using sentence simplification and audio acceleration, if the video still cannot be aligned, we extract the lengthy audio, subtitles, and corresponding video clips, and then slow down the video speed to align it properly.
By simplifying sentences, accelerating audio, and slowing down videos, we can truly achieve alignment of the new video's sound, picture, and subtitles.
Of course, these processing methods may also produce other issues, such as reduced translation accuracy (resulting from sentence simplification), audio sounding too fast, and video appearing slow. However, you should know that even if you use manual translation and video editing, you will encounter similar issues. But the cost and time required to solve these problems would be much higher. and AI will do better than most of us. GhostCut do best in this job.