#I'm failing so far | Explore Tumblr Posts and Blogs

a-memory-a-distant-echo · 3 months

Text

it's time for the long-threatened post about how to get subtitles (including translated ones) for videos that don't have subtitles.

in my experience, the methods in this post can probably get you solidly 75% or more of the content of many videos (caveats inside). i've tested this on videos that are originally in chinese, english, french, german, hindi, japanese, korean, spanish, and honestly probably some languages that i'm forgetting. my experience is that it works adequately in all of them. not great, necessarily, but well enough that you can probably follow along.

this is a very long post because this is the overexplaining things website, and because i talk about several different ways to get the captions. this isn't actually difficult, though, or even especially time consuming—the worst of it is pushing a button and ignoring things for a while. actual hands-on work is probably five minutes tops, no matter how long the video is.

i've attempted to format this post understandably, and i hope it's useful to someone.

first up, some disclaimers.

this is just my experience with things, and your experience might be different. the tools used for (and available for) this kind of thing change all the time, and if you're reading this six months after i wrote it, your options might be different. this post is probably still a decent starting place.

background about my biases in this: i work in the creative industries. mostly i'm a fiction editor. i've also been a writer, a technical editor and writer, a transcriptionist, a copyeditor, and something i've seen called a 'translation facilitator' or 'rewrite editor', where something is translated fairly literally (by a person or a machine) and then a native speaker of the target language goes through and rewrites/restructures as needed to make the piece read more naturally in the target language. i've needed to get information out of business meetings that were conducted in a language i didn't speak, and have done a lot of work on things that were written in (or translated into) the writer's second or sixth language, but needed to be presented in natural english.

so to start, most importantly: machine translation is never going to be as good as a translation done by an actual human. human translators can reflect cultural context and nuanced meanings and the artistry of the work in a way that machines will never be able to emulate. that said, if machine translation is your only option, it's better than nothing. i also find it really useful for videos in languages where i have enough knowledge that i'm like, 75% sure that i'm mostly following, and just want something that i can glance at to confirm that.

creating subs like this relies heavily on voice-to-text, which—unfortunately—works a lot better in some situations than it does in others. you'll get the best, cleanest results from videos that have slow, clear speech in a 'neutral' accent, and only one person speaking at a time. (most scripted programs fall into this category, as do many vlogs and single-person interviews.) the results will get worse as voices speed up, overlap more, and vary in volume. that said, i've used this to get captions for cast concerts, reality shows, and variety shows, and the results are imperfect but solidly readable, especially if you have an idea of what's happening in the plot and/or can follow along even a little in the broadcast language.

this also works best when most of the video is in a single language, and you select that language first. the auto detect option sometimes works totally fine, but in my experience there's a nonzero chance that it'll at least occasionally start 'detecting' random other languages in correctly, or someone will say a few words in spanish or whatever, but the automatic detection engine will keep trying to translate from spanish for another three minutes, even tho everything's actually in korean. if there's any way to do so, select the primary language, even if it means that you miss a couple sentences that are in a different language.

two places where these techniques don't work, or don't work without a lot of manual effort on your part: translating words that appear on the screen (introductions, captions, little textual asides, etc), and music. if you're incredibly dedicated, you can do this and add it manually yourself, but honestly, i'm not usually this dedicated. getting captions for the words on the screen will involve either actually editing the video or adding manually translated content to the subs, which is annoying, and lyrics are...complicated. it's possible, and i'm happy to talk about it in another post if anyone is interested, but for the sake of this post, let's call it out of scope, ok? ok. bring up the lyrics on your phone and call it good enough.

places where these techniques are not great: names. it's bad with names. names are going to be mangled. resign yourself to it now. also, in languages that don't have strongly gendered speech, you're going to learn some real fun stuff about the way that the algorithms gender things. (spoiler: not actually fun.) bengali, chinese, and turkish are at least moderately well supported for voice-to-text, but you will get weird pronouns about it.

obligatory caveat about ai and voice-to-text functionality. as far as i'm aware, basically every voice-to-text function is ~ai powered~. i, a person who has spent twenty years working in the creative industries, have a lot of hate for generative ai, and i'm sure that many of you do, too. however, if voice-to-text (or machine translation software) that doesn't rely on it exists anymore, i'm not aware of it.

what we're doing here is the same as what douyin/tiktok/your phone's voice-to-text does, using the same sorts of technology. i mention this because if you look at the tools mentioned in this post, at least some of them will be like 'our great ai stuff lets you transcribe things accurately', and i want you to know why. chat gpt (etc) are basically glorified predictive text, right? so for questions, they're fucking useless, but for things like machine transcription and machine translation, those predictions make it more likely that you get the correct words for things that could have multiple translations, or for words that the software can only partially make out. it's what enables 'he has muscles' vs 'he has mussels', even though muscles and mussels are generally pronounced the same way. i am old enough to have used voice to text back when it was called dictation software, and must grudgingly admit that this is, in fact, much better.

ok! disclaimers over.

let's talk about getting videos

for the most part, this post will assume that you have a video file and nothing else. cobalt.tools is the easiest way i'm aware of to download videos from most sources, though there are other (more robust) options if you're happy to do it from the command line. i assume most people are not, and if you are, you probably don't need this guide anyhow.

i'm going to use 'youtube' as the default 'get a video from' place, but generally speaking, most of this works with basically any source that you can figure out how to download from—your bilibili downloads and torrents and whatever else will work the same way. i'm shorthanding things because this post is already so so long.

if the video you're using has any official (not autogenerated) subtitles that aren't burned in, grab that file, too, regardless of the language. starting from something that a human eye has looked over at some point is always going to give you better results. cobalt.tools doesn't pull subtitles, but plugging the video url into downsub or getsubs and then downloading the srt option is an easy way to get them for most places. (if you use downsub, it'll suggest that you download the full video with subtitles. that's a link to some other software, and i've never used it, so i'm not recommending it one way or the other. the srts are legit, tho.)

the subtitle downloaders also have auto translation options, and they're often (not always) no worse than anything else that we're going to do here—try them and see if they're good enough for your purposes. unfortunately, this only works for things that already have subtitles, which is…not that many things, honestly. so let's move on.

force-translating, lowest stress mode.

this first option is kind of a cheat, but who cares. youtube will auto-caption things in some languages (not you, chinese) assuming that the uploader has enabled it. as ever, the quality is kinda variable, and the likelihood that it's enabled at all seems to vary widely, but if it is, you're in for a much easier time of things, because you turn it on, select whatever language you want it translated to, and youtube…does its best, anyhow.

if you're a weird media hoarder like me and you want to download the autogenerated captions, the best tool that i've found for this is hyprscribr. plug in the video url, select 'download captions via caption grabber', then go to the .srt data tab, copy it out, and paste it into a text file. save this as [name of downloaded video].[language code].srt, and now you have captions! …that you need to translate, which is actually easy. if it's a short video, just grab the text, throw it in google translate (timestamps and all), and then paste the output into a new text file. so if you downloaded cooking.mp4, which is in french, you'll have three files: cooking.mp4, cooking.fr.srt, and cooking.en.srt. this one's done! it's easy! you're free!

but yeah, ok, most stuff isn't quite that easy, and auto-captioning has to be enabled, and it has some very obvious gaps in the langauges it supports. which is sort of weird, because my phone actually has pretty great multilingual support, even for things that youtube does not. which brings us to low-stress force translation option two.

use your phone

this seems a little obvious, but i've surprised several people with this information recently, so just in case. for this option, you don't even need to have downloaded the video—if it's a video you can play on your phone, the phone will almost definitely attempt real-time translation for you. i'm sure iphones have this ability, but i'm an android person, so can only provide directions for that: go into settings and search for (and enable) live translation. the phone will do its best to pick up what's being said and translate it on the fly for you, and if 'what's being said' is a random video on the internet, your phone isn't gonna ask questions. somewhat inexplicably, this works even if the video is muted. i do this a lot at like four a.m. when i'm too lazy to grab earbuds but don't want to wake up my wife.

this is the single least efficient way to force sub/translate things, in my opinion, but it's fast and easy, and really useful for those videos that are like a minute long and probably not that interesting, but like…what if it is, you know? sometimes i'll do this to decide if i'm going to bother more complicated ways of translating things.

similarly—and i feel silly even mentioning this, but that i didn't think of it for an embarrassingly long time—if you're watching something on a device with speakers, you can try just…opening the 'translate' app on your phone. they all accept voice input. like before, it'll translate whatever it picks up.

neither of these methods are especially useful for longer videos, and in my experience, the phone-translation option generally gives the least accurate translation, because in attempting to do things in real time, you lose some of the predicative ability that i was talking about earlier. (filling in the blank for 'he has [muscles/mussels]' is a lot harder if you don't know if the next sentence is about the gym or about dinner.)

one more lazy way

this is more work than the last few options, but often gives better results. with not much effort, you can feed a video playing on your computer directly into google translate. there's a youtube video by yosef k that explains it very quickly and clearly. this will probably give you better translation output than any of the on-the-fly phone things described above, but it won't give you something that you can use as actual subs—it just produces text output that you can read while you watch the video. again, though, really useful for things that you're not totally convinced you care about, or for things where there aren't a lot of visuals, or for stuff where you don't care about keeping your eyes glued to the screen.

but probably you want to watch stuff on the screen at the same time.

let's talk about capcut!

this is probably not a new one for most people, but using it like this is a little weird, so here we go. ahead of time: i'm doing this on an actual computer. i think you probably can do it on your phone, but i have no idea how, and honestly this is already a really long guide so i'm not going to figure it out right now. download capcut and put it on an actual computer. i'm sorry.

anyhow. open up capcut, click new project. import the file that you downloaded, and then drag it down to the editing area. go over to captions, auto captions, and select the spoken language. if you want bilingual captions, pick the language for that, as well, and the captions will be auto-translated into whatever the second language you choose is. (more notes on this later.)

if i remember right, this is the point at which you get told that you can't caption a video that's more than an hour long. however. you have video editing software, and it is open. split the video in two pieces and caption them separately. problem solved.

now the complicated part: saving these subs. (don't panic; it's not actually that complicated.) as everyone is probably aware, exporting captions is a premium feature, and i dunno about the rest of you, but i'm unemployed, so let's assume that's not gonna happen.

the good news is that since you've generated the captions, they're already saved to your computer, they're just kinda secret right now. there are a couple ways to dig them out, but the easiest i'm aware of is the biyaoyun srt generator. you'll have to select the draft file of your project, which is auto-saved once a minute or something. the website tells you where the file is saved by default on your computer. (i realised after writing this entire post that they also have a step-by-step tutorial on how to generate the subtitles, with pictures, so if you're feeling lost, you can check that out here.)

select the project file titled 'draft_content', then click generate. you want the file name to be the same as the video name, and again, i'd suggest srt format, because it seems to be more broadly compatible with media players. click 'save to local' and you now have a subtitle file!

translating your subtitles

you probably still need to translate the subtitles. there are plenty of auto-translation options out there. many of them are fee- or subscription-based, or allow a very limited number of characters, or are like 'we provide amazing free translations' and then in the fine print it says that they provide these translations through the magic of uhhhh google translate. so we're just going to skip to google translate, which has the bonus of being widely available and free.

for shorter video, or one that doesn't have a ton of spoken stuff, you can just copy/paste the contents of the .srt file into the translation software of your choice. the web version of google translate will do 5000 characters in one go, as will systran. that's the most generous allocation that i'm aware of, and will usually get you a couple minutes of video.

the timestamps eat up a ton of characters, though, so for anything longer than a couple minutes, it's easier to upload the whole thing, and google translate is the best for that, because it is, to my knowledge, the only service that allows you to do it. to upload the whole file, you need a .doc or .rtf file.

an .srt file is basically just a text file, so you can just open it in word (or gdocs or whatever), save it as a .doc, and then feed it through google translate. download the output, open it, and save it as an .srt.

you're done! you now have your video and a subtitle file in the language of your choice.

time for vibe, the last option in this post.

vibe is a transcription app (not a sex thing, even tho it sounds like one), and it will also auto-translate the transcribed words to english, if you want.

open vibe and select your file, then select the language. if you want it translated to english, hit advanced and toggle 'translate to english'. click translate and wait a while. after a few minutes (or longer, depending on how long the file is), you'll get the text. the save icon is a folder with a down arrow on it, and i understand why people are moving away from tiny floppy disks, but also: i hate it. anyhow, save the output, and now you have your subs file, which you can translate or edit or whatever, as desired.

vibe and capcom sometimes get very different results. vibe seems to be a little bit better at picking up overlapping speech, or speech when there are other noises happening; capcom seems to be better at getting all the worlds in a sentence. i feel like capcom maybe has a slightly better translation engine, of the two of them, but i usually end up just doing the translation separately. again, it can be worth trying both ways and seeing which gives better results.

special notes about dual/bilingual subs

first: i know that bilingual subs are controversial. if you think they're bad, you don't have to use them! just skip this section.

as with everything else, automatically generating gives mixed results. sometimes the translations are great, and sometimes they're not. i like having dual subs, but for stuff that Matters To Me, for whatever reason, i'll usually generate both just the original and a bilingual version, and then try some other translation methods on the original or parts thereof to see what works best.

not everything displays bilingual subs very well. plex and windows media player both work great, vlc and the default video handler on ubuntu only display whatever the first language is, etc. i'm guessing that if you want dual subbed stuff you already have a system for it.

i'll also point out that if you want dual subs and have gone a route other than capcom, you can create dual subs by pasting the translated version and the untranslated version into a single file. leave the timestamps as they are, delete the line numbers if there are any (sometimes they seem to cause problems when you have dual subs, and i haven't figured out why) and then literally just paste the whole sub file for the first language into a new file. then paste in the whole sub file for the second language. yes, as a single chunk, the whole thing, right under the first language's subs. save the file as [video name].[zh-en].srt (or whatever), and use it like any other sub file.

notes on translation, especially since we're talking about lengthy machine-translations of things.

i default to translation options that allow for translating in large chunks, mostly because i'm lazy. but since an .srt is, again, literally just a text file, they're easy to edit, and if you feel like some of the lines are weird or questionable or whatever, it's easy to change them if you can find a better translation.

so: some fast notes on machine translation options, because i don't know how much time most people spend thinking about this kind of stuff.

one sort of interesting thing to check out is the bing translator. it'll only do 1000 characters at once, but offers the rather interesting option of picking a level of formality. i can't always get it to work, mind, but it's useful especially for times when you're like 'this one line sounds weird'—sometimes the difference between what the translator feels is standard vs formal vs casual english will make a big difference.

very fast illustration of the difference in translations. the random video that i used to make sure i didn't miss any steps explaining things starts with '所以你第二季来'. here's how it got translated:

google: So you come to season 2

google's top alternative: So you come in the second season

bing's standard tone: So here you come for the second season

bing set to casual: So you're coming for the second season, huh?

reverso default guess: So you come in season two

reverso alternate guess: You'll be participating in season two

capcom: So you come in season two

yandex: So you come in the second season

systran: That's why you come in season two

deepl: That's why you're here in season two

vibe: So your second season is here

technically all conveying the same information, but the vibes are very different. sometimes one translator or another will give you a clearly superior translation, so if you feel like the results you're getting are kinda crap, try running a handful of lines through another option and see if it's better.

ok! this was an incredibly long post, and i've almost definitely explained something poorly. again, there are almost certainly better ways to do this, but these ways are free and mostly effective, and they work most of the time, and are better than nothing.

feel free to ask questions and i'll answer as best i can. (the answer to any questions about macs or iphones is 'i'm so sorry, i have no idea tho.' please do not ask those questions.)

#i'm so excited to find out what i totally failed to explain because i'm sure there's something #subtitles #i really do want to reiterate that this is VERY FAR from a perfect system #but it's better than nothing #i assume that we all dream of having at least fluent comprehension of basically every language #but here in the real world...#y'know.#echoes linger

95 notes · View notes