Real-time factor (RTF and RTFx)

Real-time factor (RTF) measures how fast a speech recognition system processes audio relative to the audio's own length. An RTF of 0.1 means ten seconds of audio take one second to transcribe. Because modern GPU systems are far faster than real time, leaderboards now often quote the inverse, RTFx: an RTFx of 1,000 means a thousand seconds of audio per second of compute. Both are throughput measures. Neither tells you how long a user waits for a word to appear, which is latency.

Definition#

Given an audio file of duration T_audio that a system processes in wall-clock time T_proc:

RTF dates from the era when running at real time on a CPU was itself an achievement, and toolkits such as Kaldi report it routinely. Lower RTF is better. RTFx was popularised by the Hugging Face Open ASR Leaderboard, where RTF values had become small decimals that were hard to read. Higher RTFx is better.

Reading the numbers#

The "typical context" column is indicative only. Where a given model falls depends on its size and architecture, the accelerator, and above all how much work is batched together.

What affects RTF#

Hardware and precision#

The same model can differ by an order of magnitude between a laptop CPU and a data-centre GPU. Running weights and activations in 16-bit or 8-bit formats rather than 32-bit increases throughput on hardware that supports them. See model size and quantisation and GPU inference for speech.

Batch size#

A GPU processing one ten-second clip is mostly idle, waiting on memory transfers and kernel launches. Processing 64 clips at once uses the same weights for all of them and fills the hardware. Throughput-oriented benchmarks run large batches, which is why headline RTFx numbers are far higher than what a single request sees.

Architecture and decoding#

Encoder-only models with CTC decoding produce output in one parallel pass. Autoregressive encoder-decoder models generate one token at a time, so their decoding cost grows with transcript length, and beam search multiplies it. Transducers sit in between. See ASR model architectures. Encoder design matters too: architectures that downsample audio more aggressively early on, such as Fast Conformer, reduce the sequence length every later layer has to process.

Audio length and padding#

Some models process fixed-length windows. Whisper, for example, works on 30-second segments, so a 5-second clip is padded and costs close to a full window. Short-form benchmarks can therefore penalise or favour a model depending on how it handles padding.

What is timed#

Some measurements include audio decoding, resampling, voice activity detection and text post-processing; others time only the model forward pass. Network time is excluded from most benchmarks but dominates for short requests to a remote API.

Measuring RTF#

A minimal harness for a local model or an API:

import time
import soundfile as sf

def rtfx(transcribe, paths):
    audio_seconds = 0.0
    start = time.perf_counter()
    for path in paths:
        info = sf.info(path)
        audio_seconds += info.frames / info.samplerate
        transcribe(path)
    elapsed = time.perf_counter() - start
    return audio_seconds / elapsed

print(f"RTFx: {rtfx(my_transcribe, files):.1f}")

This measures sequential throughput. To measure a batched or concurrent system, submit requests in parallel and divide total audio by total wall-clock time. Warm the system up first: the first request often includes model loading, memory allocation or compilation.

RTF is not latency#

A system with RTFx of 1,000 can still make a user wait. In streaming recognition, audio arrives in real time, so a stream can never be processed faster than it is spoken. What the user feels is the delay between speaking and seeing the text, which depends on chunk size, look-ahead, endpointing and network round trips. See speech-to-text latency.

For streaming, speed beyond real time is spent on two things: keeping each chunk's processing time small, which caps the compute component of latency, and serving many streams on one GPU. A server with an aggregate RTFx of 200 can, in principle, keep up with 200 simultaneous live streams, before headroom for bursts. See concurrency and scaling.

For batch transcription, RTF matters more directly. It sets how quickly a backlog clears and, through GPU hours, what transcription costs.

How Reson8 approaches this#

Resonant-1 is served with Violin, Reson8's own inference engine for speech, which uses fused kernels, CUDA graphs and decoding scheduling designed for speech workloads. On the Open ASR Leaderboard short-form English sets (April 2026), Resonant-1 recorded an RTFx of 1187 alongside a 6.51% average WER. At that rate, an hour of audio takes about three seconds under the leaderboard's batched conditions. Live streams remain bound by real time; for voice agents, the relevant figure is the 326 ms median time from end of speech to final transcript on the Pipecat benchmark. Details are in the benchmarks post.

Common questions#

References#

1. Srivastav, V. et al. Open ASR Leaderboard. Hugging Face.
2. Povey, D. et al. (2011). The Kaldi Speech Recognition Toolkit. IEEE ASRU Workshop.
3. Radford, A. et al. (2022). Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). arXiv:2212.04356.
4. Gulati, A. et al. (2020). Conformer: Convolution-augmented Transformer for Speech Recognition. Interspeech.
5. Rekesh, D. et al. (2023). Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition. IEEE ASRU Workshop.