hi, all, Today,I will introduce the background and technical principles of Hard Sub Translation product. The technology concept that I most want to share with everyone will be mentioned later.
As we all know, hard subtitles are embedded in the video and the subtitles are already encoded in the video. Many video translation software extract the sound in the video using ASR technology, and then translate it. But many videos have background music and subtitles, how can these videos be translated? This is actually a problem in the market, so there are currently no good hard subtitle extraction and editing software on the market.
Due to the fact that hard subtitles are already embedded in the video, OCR technology is usually used. However, if we want to annotate the text in the video as before, it will be very complicated. Image annotation does not have a time concept, so just draw a box. But video is continuous, with time and space concepts, and when we annotate, there may be several situations to consider:
Therefore, hard subtitle extraction requires more automation, higher recognition accuracy, and text grouping, because the text style of the same group is consistent.
After translation, layout and alignment also need to be considered, because the translated text may be long or short.
Inpainting needs to be combined with hard subtitle extraction. Usually, we have a requirement to put the new text in exactly the same location as the original subtitle. This requires Inpainting of the original video. Therefore, the efficiency and effect of Inpainting also need to be considered.
Let's summarize the difficulties of hard subtitle translation:
Our GhostCut hard subtitle translation uses some of the technologies mentioned above to achieve fully automatic translation of hard subtitles. Fully automatic translation may have some problems, including extraction errors, translation errors, etc., so an online editor needs to be provided to facilitate adjustments and modifications.
Okay, the most important thing is here. What we really want to share is this product concept. After abstraction, it is actually: