How VEED keeps subtitles in sync with the word, on every video its users upload

Video editing runs on timing accuracy. VEED transcribes many of the videos its users upload with Reson8, turning speech into word-level timestamps to drive animated subtitles. VEED brought in Reson8 for one reason: to deliver timestamp accuracy so that audio, video, and text align perfectly.

At a glance

A lagged subtitle leads to a failed edit

VEED (veed.io) is an AI-powered online video editor, powering millions of teams across the world’s leading companies to create high-quality content and ads across video, audio, and text. For every video uploaded, the transcript is essential to the edit; it decides when a subtitle appears, where a clip is cut, and which words are highlighted as they’re spoken.

For video editing, the hard part of transcription is not getting the words right, it’s getting the timing right. Even if a word is perfectly transcribed, if it appears in a video a fraction of a second too early or too late, the mismatch is felt by viewers. This requirement becomes even stricter when you cut videos on these timings; VEED strips filler words automatically and needs the exact start and end of each ‘um’ or ‘uh’ so only the filler word is cut out, and nothing else.

VEED runs several speech-to-text providers and benchmarks them on word-level timestamp deviation first and foremost, because that’s where the quality of the product lives.

“For everything we build, timestamp precision matters more than the word error rate. A transcript that reads well but drifts by a few frames still gives our users a bad edit. That was the bar we tested Reson8 against.”Kieran Dunbar, Senior API Engineer, VEED

Tuning the timestamps to how VEED measures them

VEED evaluated Reson8 the same way it benchmarks any other provider: real clips, its own methodology, scored on word-level timestamp deviation. In the first pass, Reson8’s timestamps were pulled towards the start of each utterance, leading to words being highlighted slightly earlier.

Together, VEED and Reson8 implemented gravity-weighted timestamping, where the timestamp is pulled towards the center of the spoken word rather than the start of the phrase. This way, in a VEED video, a word lights up exactly as it is spoken and clears immediately on the silence after.

Filler words were given the same treatment. Giving each word a fixed length would have led to most filler words being left in on a cut, so Reson8 implemented real per-word durations. As Reson8 runs everything on its own infrastructure, the change was made on the same day VEED brought it up. Both changes live in Reson8’s model and improve as it trains, so every Reson8 customer benefits.

“We transcribe every upload, and we benchmark providers constantly. Reson8 performed exceptionally well on the one thing our editing features depend on, word-level timing. They moved at the speed we do, and we’re expanding our partnership as a result. We’re glad to be partnering with Reson8 to make VEED’s product offering better.”Josh Goldman, COO, VEED

For VEED, timestamp accuracy was the reason to bring Reson8 in. Because Reson8 treats word-level timing as a target it can move on, the teams were able to tune timestamping in line with VEED’s requirements. Today, VEED runs transcription on Reson8’s Netherlands-based infrastructure, with zero audio retention and no training on VEED data.

Editing on top of transcripts, where even the slightest drift breaks the result? Run your real clip audio through Reson8 and compare word-level timestamp deviation against your current engine. Get in touch or get started at console.reson8.dev.