Subtitle Tool Req. (w/local STT / ASR) - Issues w/SubtitleEdit + vLLM

Hey folks, this is Translation-adjacent (and part of my translation workflow). I'm wondering if anyone has had success using the "OpenAI-Compatible API" functionality in SubtitleEdit with e.g. vLLM. I usually get 0 Segments Decoded, despite vLLM apparently outputing quite a few tokens.

I'm hoping to use a local model for Danish ASR, and there are a few I can run with vLLM, but I've yet to have success with any of them, unfortunately! What - if anything - are other folks using for workflows like this?

reddit.com
u/overand — 13 days ago
▲ 3 r/machinetranslation+1 crossposts

The Great Knit-Off eng, subtitle project (Translation Comparison, etc for Den Store Strikkedyst)

Hello! This is a controversial topic, so, I'll start with the disclaimers:

  1. Nothing is better than a human translator.
  2. Everyone deserves to have a living wage, and LLMs / machine learning / AI is (in many parts of the world) damage people's abilities to have a living wage.
  3. I have a budget of $0 / 0 kr. / € 0
  4. I run all of my AI workloads locally, on my own computer hardware.

Hey folks! I'm working on a personal project, as of yet shared only with one in-person friend, to create English subtitles for Den Store Strikkedyst, aka "The Great Knit-Off." Only a few episodes are available with english subtitles. Youtube doesn't have machine-subtitles for Danish, and even the official tvsyd site doesn't have subtitles for most.

I've been using some machine-learning / AI tools to both generate subtitles for these episodes in Danish, and to translate to English. Details on the technical workflow are in a reply.

I have guesses about which LLMs are better/worse for this, but I'd like some feedback on these transcriptions and translations if possible! (I could upload segments with subtitles for folks to view, but I don't know the copyright status of this; it's from 2014.)

I have no interest in profiting on this, and don't want to take work away from translators. (If anyone wants to volunteer to help with subtitling and translation, I'm certainly open to it!)

The following is from the intro of Season 3 Episode 1.

Original Text (via Whisper-Large-V3, I think)

[00:06.830] Den store strikkedyst er endelig tilbage.
[00:10.850] 10 dygtige strikkere er blevet udvalgt til at dyste i sprit nye opgaver.
[00:15.470] De skal være super skarpe de næste par uger, for kun én af dem går hjem som vinder.
[00:20.650] Og denne benhårde strikkekonkurrence, den begynder... nu!
[00:28.240] 10 udvalgte strikkere, så klar til verdens hårdeste strikkekonkurrence.
[00:33.120] Station 3 kommer i nul.
[00:34.700] Jeg er bare klar til at komme ind og drikke.
[00:36.240] Jeg regner med, at det skal nok gå.
[00:38.600] Jeg er helt sikker på, at jeg vinder.
[00:40.000] Jeg går efter at vinde. Det gør jeg helt sikkert.
[00:44.620] Jeg har været spændt på at møde de andre deltagere og se, hvad de er for nogen.
[00:47.660] Hvordan ser de ud? Hvor gammel er de? Er de lige så spændte som mig?
[00:52.400] Det kommer til at blive på nogle wildcards og noget held og noget kreativitet.
[00:56.500] Jeg ved ikke lige, hvor jeg kommer fra.
[00:58.760] Men jeg håber da, at jeg vinder. Selvfølgelig gør jeg det.

LLM A

[00:06.830] The Great Knitting Competition is finally back.
[00:10.850] 10 skilled knitters have been chosen to compete in brand new tasks.
[00:15.470] They need to be super sharp over the next couple of weeks, because only one of them will go home as the winner.
[00:20.650] And this cutthroat knitting competition begins... now!
[00:28.240] 10 selected knitters, all ready for the world's toughest knitting competition.
[00:33.120] Station 3 is resetting.
[00:34.700] I'm just ready to get in there and dive in.
[00:36.240] I reckon it'll go alright.
[00:38.600] I'm absolutely sure that I'll win.
[00:40.000] I'm going for the win. I definitely am.
[00:44.620] I've been excited to meet the other participants and see what they're like.
[00:47.660] What do they look like? How old are they? Are they as excited as I am?
[00:52.400] It's going to come down to some wildcards, some luck, and some creativity.
[00:56.500] I'm not quite sure where I stand.
[00:58.760] But I do hope I win. Of course I do.

LLM B

[00:06.830] The great knitting showdown is finally back.
[00:10.850] 10 skilled knitters have been selected to compete in brand new challenges.
[00:15.470] They will have to be super sharp over the next couple of weeks, because only one of them will go home as the winner.
[00:20.650] And this bone-hard knitting competition, it begins... now!
[00:28.240] 10 selected knitters, ready for the world's hardest knitting competition.
[00:33.120] Station 3, zero in.
[00:34.700] I'm just ready to go in and get started.
[00:36.240] I figure it will turn out fine.
[00:38.600] I am absolutely sure that I will win.
[00:40.000] I'm going for the win. I'm doing it for sure.
[00:44.620] I've been looking forward to meeting the other contestants and seeing what they're like.
[00:47.660] What do they look like? How old are they? Are they as excited as me?
[00:52.400] It's going to come down to some wildcards, some luck, and some creativity.
[00:56.500] I don't really know where I stand.
[00:58.760] But I do hope I win. Of course I do.

LLM C

[00:06.830] The great knitting duel is finally back.
[00:10.850] 10 skilled knitters have been selected to compete in brand-new tasks.
[00:15.470] They need to be super sharp over the next few weeks, because only one of them will go home as the winner.
[00:20.650] And this hardcore knitting competition begins... now!
[00:28.240] 10 selected knitters, all ready for the world's toughest knitting competition.
[00:33.120] Station 3 is starting at zero.
[00:34.700] I'm just ready to get in there and drink.
[00:36.240] I expect it's going to go well.
[00:38.600] I am completely sure that I will win.
[00:40.000] I'm going for the win. I'm definitely going to do that.
[00:44.620] I've been excited to meet the other participants and see what they're like.
[00:47.660] What do they look like? How old are they? Are they as excited as I am?
[00:52.400] It's going to come down to some wildcards, some luck, and some creativity.
[00:56.500] I don't really know where I stand.
[00:58.760] But I hope that I win. Of course I do.

LLM D

[00:06.830] The Great Knitting Duel is finally back.
[00:10.850] 10 skilled knitters have been selected to compete in brand-new challenges.
[00:15.470] They need to be incredibly sharp over the next few weeks, because only one of them will go home as the winner.
[00:20.650] And this hardcore knitting competition begins... now!
[00:28.240] 10 chosen knitters, ready for the world's toughest knitting competition.
[00:33.120] Station 3 is underway.
[00:34.700] I'm just ready to get in there and drink.
[00:36.240] I expect it'll go well.
[00:38.600] I'm absolutely sure that I'll win.
[00:40.000] I'm going for the win. I'm absolutely certain of it.
[00:44.620] I've been excited to meet the other contestants and see who they are.
[00:47.660] What do they look like? How old are they? Are they just as excited as I am?
[00:52.400] It's going to come down to some wildcards, a bit of luck, and some creativity.
[00:56.500] I don't really know what to expect.
[00:58.760] But I do hope I win. Of course I do.

LLM E

[00:06.830] The Great Knitting Contest is finally back.
[00:10.850] 10 skilled knitters have been selected to compete in brand new tasks.
[00:15.470] They will have to be super sharp over the next few weeks, because only one of them will go home as the winner.
[00:20.650] And this cutthroat knitting competition begins... now!
[00:28.240] 10 selected knitters, all ready for the world's toughest knitting competition.
[00:33.120] Station 3 is at zero.
[00:34.700] I'm just ready to get in there and drink. (Note: Likely a slip of the tongue or slang in the original audio)
[00:36.240] I reckon it'll go alright.
[00:38.600] I am absolutely sure that I will win.
[00:40.000] I'm going for the win. I definitely am.
[00:44.620] I've been excited to meet the other participants and see what they're like.
[00:47.660] What do they look like? How old are they? Are they as excited as I am?
[00:52.400] It's going to come down to some wildcards, some luck, and some creativity.
[00:56.500] I'm not quite sure where I stand.
[00:58.760] But I do hope that I win. Of course I do.

(I'll reveal which LLMs these were later on, if desired)

One of the areas of difference is, in "Og denne benhårde strikkekonkurrence," I see translations as "cutthroat," "bonehard," and "hardcore."

As a native US English speaker, I'd say the following:

  • cutthroat - This implies intense competition between people - as in, one might (metaphorically) need to cut the throat (murder) an opponent to succeed.
  • bone-hard - Nope - I've never heard this used.
  • hardcore - this more specifically means "very difficult" - it's not specific to competitions. It can have an adult / NSFW / pornographic connotation, as "hardcore" is a style of pornography, but it's also used to refer to very challenging difficulty video games, for example.

I'd love any feedback people have about this project, and any help folks would like to offer, technically or otherwise.

u/overand — 14 days ago

Was kinda interested in trying Marinara Engine, but in reviewing the git commits...

https://preview.redd.it/4u7s9xs5061h1.png?width=646&format=png&auto=webp&s=94357efe8f94ef1c7f4839b867f496ebcf329fef

https://preview.redd.it/g4karki7061h1.png?width=838&format=png&auto=webp&s=05401a8ecba7446a71bafded463b1e02e23af395

bruh

(images for context - not visible in previews in all reddit clients)

BUT, in seriousness: nice work on the project in general! And in other seriousness, please for the love of god don't put brainrot in commit history! It might give aneurisms to us aging millennials. And those Gen X folks. And Boomers.

Actually, on second thought... maybe you should keep doing it.

(edit: removed an edit mistake, sorry for breaking any jokes about repeating myself, heh)

reddit.com
u/overand — 3 months ago