Meet Rumble-2
Rumble-2 is one of the world's most accurate speech-to-text models, and is now available across 50+ languages. It leads across English benchmarks with lowest streaming WER and p95 latency, and across medical benchmarks, outperforming medical specific models.
We built it for products with voice at its core such as meeting notetakers, clinical documentation and voice agents. Now live in the API as rumble-2.
Below you will find an overview of Rumble-2 across internal and external benchmarks, we aim to provide a comprehensive and multi-faceted view on its performance.
English benchmarks
Speech benchmarks are great, but many of them lean on clean, carefully recorded speech, and publicly available datasets that models can train on. That makes it hard to tell which model will hold up on the real-world audio you run into.
We turned to Cekura, Coval and Bluejay, three independent benchmarks for speech models. They choose the audio, test every model the same way and publish the results.
| Model | Cekura, Converse STT WER | Coval, streaming WER | Coval, p95 latency | Bluejay, English WER (batch) |
|---|---|---|---|---|
| Rumble-2 | 2.05% | 2.1% | 54 ms | 3.41% |
| AssemblyAI Universal | 2.71%¹ | 2.4% | 137 ms | 3.78% |
| ElevenLabs Scribe v2 | — | 4.3%² | 148 ms² | 3.60% |
| Deepgram Nova-3 | 3.78% | — | — | — |
| Soniox STT-RT v5 | 3.02% | 4.6%³ | 91 ms³ | — |
Cekura Bench
Rumble-2 ranks first on overall WER at 2.05%, ahead of Fish Audio Transcribe Stream 1 (2.21%) and NVIDIA Nemotron 3.5 Transcribe (2.28%). That’s 24% fewer errors than AssemblyAI Universal 3.6 Pro (2.71%) and 46% fewer than Deepgram Nova-3 (3.78%).
Coval
Coval benchmarks streaming models on both speed and accuracy. While most models gain one by giving up the other, Rumble-2 comes out on top on both. Out of 22 models, it has the lowest latency (54 ms p95 time to final segment) and the lowest word error rate (1.7%). The next fastest, Qwen3 ASR Fast, takes 77 ms at 2.5% WER.
No other model on Coval’s chart beats Rumble-2 on either axis.
Bluejay MISTT
On Bluejay’s English benchmark, Rumble-2 Realtime ranks second among streaming models at 3.55%, behind MAI-Transcribe-2 Streaming (3.43%). In batch, Rumble-2 ranks third at 3.41%, behind MAI-Transcribe-2 (3.33%) and Smallest Pulse Pro (3.38%). The top three batch results fall within each other’s 95% confidence intervals.
AppTek
We also ran our own tests on a larger set of real calls: AppTek’s Call-Center Dialogues, 302 hours of customer-service calls across 14 English accents. We scored every model with AppTek’s own script, so the text normalization is theirs, not ours. Rumble-2 made 9% fewer errors than NVIDIA’s Nemotron-3.5-Transcribe (87,285 vs. 96,335) and 23% fewer than Voxtral Small Transcribe. Most of Voxtral’s gap comes from words it dropped entirely: 50,943 deletions against our 18,942.
“For us, timestamp precision matters more than word error rate, because our users feel it in the captions and word highlighting. With Rumble-2, the words land where they're spoken.”
.png)
Kieran Dunbar
Senior API Engineer, VEED
Turn detection
With voice agents, the delay a caller notices isn’t transcription time. It’s the gap between when they stop talking and when the agent answers.
Most providers decide when a turn ends with a fixed silence timer. Set it short and the agent cuts people off mid-thought. Set it long and every reply starts with dead air.
Rumble-2 makes the end-of-turn decision inside the speech model, from what was said and how, not from silence alone. “I’d like to move my appointment to…” is unfinished no matter how long the pause. “…next Tuesday, if possible.” is finished.
On top of that:
- Early end-of-turn candidates let your agent start preparing a reply before the caller finishes.
- Interruptions are only flagged once there are actual words, so a cough or background noise won’t cut the agent off.
Jack & Jill, the AI recruitment platform, runs 15- to 20-minute candidate interviews through its voice agent. After switching to Reson8’s turn detection, Jack & Jill saw turns per call drop by 50% and utterances get 50% longer, with no change in total call length. Candidates stopped getting cut off halfway through an answer and gave the agent much fuller answers about their experience.
Multilingual: much more than English
Open ASR Leaderboard
The Open ASR Leaderboard is a public benchmark on Hugging Face, with fixed test sets and scoring code anyone can rerun. Its multilingual track covers German, French, Italian, Spanish, Portuguese and Dutch across FLEURS, Common Voice and Multilingual LibriSpeech, plus Hindi on Voice Arena Monsoon. We ran Rumble-2 on the same test set using the leaderboard’s own scoring.
Averaged over the seven languages, Rumble-2 scores 4.35% WER. Against the leaderboard’s top 11 as of November 4, that puts it third. It’s level with OpenAI’s gpt-transcribe (4.36%), behind ElevenLabs Scribe v2 (3.56%) and Microsoft Azure Speech (4.14%), and ahead of the other eight, including AssemblyAI, NVIDIA’s 30B Nemotron and Mistral’s Voxtral. It stays under 5% in all six European languages, from 2.47% in Spanish to 4.39% in Portuguese. Hindi is the hardest, at 8.95%.
Medical
WER counts a dropped “the” and a misheard “metoprolol” as one error each. Only one changes what a doctor reads. WER is still the metric every provider publishes, so it’s what you can compare.
We tested on PriMock57 (mock primary-care consultations, English) and MedDictate (clinical dictation in English, French and German), against ElevenLabs.
| Rumble-2 | ElevenLabs | Fewer errors | |
|---|---|---|---|
| PriMock57 | 8.13% | 9.72% (Scribe v2) | 16% |
| MedDictate, English | 2.34% | 3.0% (Scribe v2 Medical) | 22% |
| MedDictate, French | 5.72% | 8.3% (Scribe v2 Medical) | 31% |
| MedDictate, German | 6.74% | 7.2% (Scribe v2 Medical) | 6% |
A note on how we tested: we didn’t run the ElevenLabs tests ourselves. The MedDictate bars are ElevenLabs’ own published Scribe v2 Medical results. ElevenLabs hasn’t published a Scribe v2 Medical result for PriMock57, so that comparison is against Scribe v2 from Omi Health’s public leaderboard, scored with the same Omi scorer we used for Rumble-2.
No benchmark knows your formulary or your clinic’s abbreviations, and every practice uses its own terms. That’s why you can give Rumble-2 up to 50,000 custom terms as plain text, with no fine-tuning or audio samples needed.
The same need shows up outside healthcare. Obsidian, an all-in-one platform for financial advisors, transcribes client meetings full of portfolio values, fees and account references, where “40,000” heard as “14,000” can be a costly mistake. They tested Reson8 on their own meeting audio against their existing stack, and now run Reson8 inside their own AWS environment.
Custom models: your words, not ours
The words that break transcription in the real world are the ones the model hasn’t trained on, like your product names, your medication list, or your customers’ last names. No public benchmark can measure accuracy on those, so we won’t pretend we can.
What Rumble-2 gives you is a way to teach the model those words. You create a custom model once, add your terms as plain text, and pass its ID with each request. If you only have a few terms, you can skip the custom model and send them along with the request itself, up to 250 per request. Rumble-2 adapts to your terminology with no fine-tuning or audio data required, so a new term takes seconds to add and is part of the next request. How hard the model leans on your list is adjustable per request.
SafeGPT uses Reson8’s custom models to give each of its care and government customers a model that transcribes their own vocabulary accurately, with up to 50,000 terms per customer.
Push any model too hard toward a word list and it starts hearing those words where nobody said them. That’s why we’d test it from both ends. Pick a recording your current engine struggles with and run it twice, once with your word list and once without. You want more of your terms coming through correctly, and you don’t want the rest of the transcript getting worse. The overall WER tells you that second part. The custom models docs walk through the setup, and if you’d rather not do it alone, send us the audio and we’ll run the comparison with you.
Run it on your own audio
A public benchmark tells you which models are worth testing. Only your own audio tells you which one to use. Take 30 to 60 minutes of real recordings, transcribe them by hand once, and score each model the same way. Next to WER, count the errors you care about: names, numbers, terms, interruptions.
Rumble-2 is live in the API as rumble-2. Sign up or book a call if you want us to run the evaluation on your data with you.
If you benchmark Rumble-2 and it loses somewhere, we want to hear about that too: commercial@reson8.dev