Real-time transcription processes audio while someone is speaking and returns text within a fraction of a second. Batch transcription processes a finished recording and returns the whole transcript once it is done. Streaming is necessary whenever text must be acted on during the conversation, as in voice agents and live captions. Batch is usually more accurate and cheaper, because the model sees the whole recording and the work can be scheduled for maximum throughput.
Key takeaways
- Choose by when the text is needed, not by how the audio arrives. A live call that is only analysed afterwards can be transcribed in batch.
- Streaming models see little or no future audio, so on the same recording they are typically somewhat less accurate than batch.
- Streaming is measured on latency; batch on throughput, usually as real-time factor.
- Streaming costs more to serve: it holds a connection and GPU capacity for the length of the audio, however quiet it is.
- Many products use both: stream for the live experience, then re-transcribe the recording in batch for the permanent record.
Two modes#
In real-time or streaming mode, the client opens a persistent connection, usually a WebSocket, and sends small chunks of audio, typically 20 to 100 ms each, as they are captured. The service returns partial results that may change as more audio arrives, and final results once a segment is settled. The mechanics are covered in streaming speech recognition.
In batch or pre-recorded mode, the client uploads a file or a URL and receives a complete transcript when processing finishes, either in the response or through a callback. Nothing is returned until the model has processed the audio, but it can process an hour of audio in far less than an hour.
Why batch is usually more accurate#
Speech is ambiguous until later words resolve it. In the patient's heart rate was ninety, the words "heart rate" constrain what the number probably is, and a later "per minute" would confirm it. A batch model uses bidirectional context: every prediction can draw on audio both before and after it. A streaming model must commit using only past audio plus a small lookahead, usually a few hundred milliseconds at most, because every millisecond of lookahead is a millisecond of added delay.
The gap depends on the model and the lookahead allowed. Models trained to run in both modes, with variable attention context, can reduce it substantially, but on equal terms full context remains an advantage. Batch processing also enables whole-file steps that streaming cannot do well:
- Diarisation that clusters voices across the entire recording rather than incrementally.
- Consistent spelling of names and terms across an hour of audio.
- Better formatting of numbers and sentences that span pauses.
- Long-context processing of long recordings, with segmentation chosen after seeing the whole file.
Different metrics#
| Real-time | Batch | |
|---|---|---|
| Primary speed metric | Latency: time from speech to text | Throughput: real-time factor, turnaround time |
| Output | Partials, then finals | One complete transcript |
| Context available | Past audio plus small lookahead | Whole recording |
| Transport | WebSocket, WebRTC or gRPC stream | HTTP upload, URL, callback |
| Failure handling | Reconnect and resume mid-stream | Retry the job |
| Serving cost | Capacity reserved for the audio's duration | Packed onto GPUs for throughput |
For streaming, the latency users feel is not one number. Time to first partial tells you how responsive live captions look; time from end of speech to final transcript tells you how quickly a voice agent can start replying. For batch, the relevant numbers are turnaround time for a typical file and cost per hour of audio.
Why streaming costs more#
A batch job can be split, batched with other jobs, and run at full GPU utilisation; an hour of audio may take well under a minute of GPU time. A streaming session arrives in real time: the service must keep state for the session, process each chunk promptly as it arrives, and hold that capacity for the full duration, including silences. Scheduling is less efficient and tail latency must be protected, so providers typically charge more per minute for streaming. See concurrency and scaling and speech-to-text pricing.
Which to use#
| Use case | Mode | Why |
|---|---|---|
| Voice agents | Real-time | The reply depends on the text within the turn |
| Live captions and subtitles | Real-time | Viewers need text while watching |
| Dictation | Real-time | Users want to see words as they speak |
| Live agent assist | Real-time | Suggestions must arrive during the call |
| Call analytics and QA | Batch | Analysed after the call; accuracy matters more |
| Media archives, podcasts | Batch | No time pressure, long files |
| Clinical notes from a recorded consultation | Batch, or both | Note is written after the visit; live view optional |
| Legal proceedings | Both | Live rough draft, certified transcript later |
Two-pass designs#
When a product needs a live experience and an accurate record, the usual design is two passes: stream for the live view, keep the audio, and re-transcribe it in batch for the stored transcript. The live and final versions will differ slightly, so the interface should make clear which one is authoritative.
How Reson8 approaches this#
Reson8 offers both modes, plus a Turns endpoint built for voice agents on top of streaming. Pricing reflects the serving cost difference: one minute of real-time streaming is 2 credits and one minute of batch is 1 credit. Domain adaptation and Patterns apply in both modes. Setup is in the documentation.
Common questions#
Can I get batch-quality accuracy in real time?
Not fully, because a streaming model cannot use audio that has not arrived yet. The gap has narrowed, and allowing slightly more lookahead or a final full-utterance pass narrows it further, at the price of latency.
Is "real-time" the same as "fast batch"?
No. A batch job on a five-second clip may return in under a second, but it still waits for the clip to be complete. Real-time means text arrives while speech is still going on.
How fast is batch transcription?
It depends on the model, the hardware and the queue. Efficient GPU-served models process audio hundreds of times faster than real time, so turnaround is often dominated by upload and queueing rather than inference.
Should I stream a file to get results sooner?
Usually not. Submitting it in batch returns the full transcript faster than streaming it at real-time speed, and with full context.
References#
- Sainath, T. N. et al. (2019). Two-Pass End-to-End Speech Recognition. Interspeech.
- He, Y. et al. (2019). Streaming end-to-end speech recognition for mobile devices. ICASSP.
- Fette, I., & Melnikov, A. (2011). The WebSocket Protocol. RFC 6455. IETF.
- Srivastav, V. et al. Open ASR Leaderboard. Hugging Face.