Integrating a speech-to-text API

Integrating a speech-to-text API means choosing between two interaction patterns, handling credentials safely, and writing code that survives the network. Pre-recorded audio goes through a request-and-response or job-based HTTP API. Live audio goes through a long-lived streaming connection, usually a WebSocket, where the client sends audio frames and receives partial and final transcripts. Most production bugs live in the streaming path: pacing, reconnection, and knowing which results are final.

Choosing batch or streaming#

The batch versus real-time choice decides most of the integration. Batch is simpler, typically cheaper, and often more accurate, because the service sees the whole recording. Streaming exists for latency: captions, agent assist, dictation and voice agents.

Batch APIs either return the transcript in the HTTP response for short files, or accept a job and deliver the result later through polling or a webhook. For webhooks, verify the signature if the provider sends one, respond quickly, and process asynchronously; retried deliveries mean your handler should be idempotent.

Authentication#

Server-to-server integrations send an API key in a header over TLS. That key must never ship in a web page or a mobile binary, where anyone can extract it. For clients that stream directly from a microphone, the usual pattern is a small backend endpoint that authenticates your user and requests a short-lived token from the provider, which the client then uses to open its connection. The alternative is to proxy audio through your own server, which costs a network hop but gives you full control over logging and routing.

A streaming session, step by step#

Details differ by provider, but the shape is nearly universal:

1. Connect over wss:// with a key or token.
2. Configure the session: audio encoding, sample rate, channel count, language, and options such as punctuation or timestamps. Some APIs take these as query parameters, others as a first JSON message.
3. Stream audio as binary frames. Chunks of 20–100 ms are typical; much larger chunks add latency, much smaller ones add overhead.
4. Receive results concurrently. Partials replace each other; finals are stable.
5. Signal the end of audio explicitly, then wait for the remaining finals before closing. Closing the socket immediately loses the last words.

A generic streaming loop in Python#

The example below uses the websockets library and a placeholder URL and message schema. It shows structure, not any specific provider's API; field names and control messages vary, so follow your provider's documentation.

import asyncio, json, os, wave
import websockets

URL = "wss://stt.example.com/v1/stream"   # placeholder
CHUNK_MS = 50

async def send_audio(ws, path):
    with wave.open(path, "rb") as wav:
        frames_per_chunk = wav.getframerate() * CHUNK_MS // 1000
        while chunk := wav.readframes(frames_per_chunk):
            await ws.send(chunk)
            await asyncio.sleep(CHUNK_MS / 1000)   # pace at real time
    await ws.send(json.dumps({"type": "end_of_stream"}))

async def receive_results(ws):
    async for message in ws:
        result = json.loads(message)
        if result.get("is_final"):
            print("FINAL  ", result["text"])
        else:
            print("partial", result["text"], end="\r")

async def transcribe(path):
    headers = {"Authorization": f"Bearer {os.environ['STT_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        await ws.send(json.dumps({"encoding": "pcm_s16le",
                                  "sample_rate": 16000, "language": "en"}))
        await asyncio.gather(send_audio(ws, path), receive_results(ws))

asyncio.run(transcribe("call.wav"))

Two things matter in this structure. Sending and receiving run as separate tasks, so a slow result never stalls the audio. And the file is paced at real time, which is what a live microphone does anyway; pushing a file as fast as possible can trip rate limits or be handled differently by the server.

Errors, retries and reconnection#

• Rate limits. HTTP 429 means slow down. Honour a Retry-After header when present.
• Transient failures. Retry 5xx responses and connection errors with exponential backoff and random jitter, capped at a sensible maximum. Do not retry 4xx errors other than 429 without changing the request; they indicate a problem with it.
• Mid-stream disconnects. Keep a short ring buffer of recently sent audio, for example the last few seconds. On reconnect, open a new session and resend from the last final result's end time, then de-duplicate overlapping text by timestamp.
• Idle timeouts. Many services close sockets that receive no audio for a while. During long silences, keep sending audio (silence is fine) or the provider's keepalive message.
• Backpressure. If the network stalls, audio from a live source keeps arriving. Bound the send queue, and decide in advance whether to drop the oldest audio or end the session, rather than letting memory and delay grow without limit.
• Session limits. Streams often have a maximum duration. For all-day streams, rotate sessions at a pause before the limit.

Production checklist#

• Declared encoding, sample rate and channel count match the actual audio. A mismatch produces garbage text, not an error. See audio formats.
• Persist finals only, keyed by stream and timestamp, so reconnections do not duplicate text.
• Log per-utterance timings on the client to track latency at p50 and p95.
• Monitor concurrent sessions against your plan's limits; see concurrency and scaling.
• Know the provider's data handling: where audio is processed, whether it is retained, and whether it is used for training. See zero data retention.

How Reson8 approaches this#

Reson8 offers real-time streaming, batch transcription of pre-recorded audio, and a Turns endpoint for voice agents that adds end-of-turn prediction and barge-in detection to the stream. Usage is credit-based: one minute of real-time streaming costs 2 credits and one minute of batch costs 1. Domain text for domain adaptation is supplied through the API and takes effect in under 60 seconds, with no retraining. Endpoints and message formats are documented at docs.reson8.dev, with account access through the console.

Common questions#

References#

1. Fette, I., & Melnikov, A. (2011). The WebSocket Protocol. RFC 6455, IETF.
2. Nottingham, M., & Fielding, R. (2012). Additional HTTP Status Codes (429 Too Many Requests). RFC 6585, IETF.
3. Fielding, R., Nottingham, M., & Reschke, J. (2022). HTTP Semantics. RFC 9110, IETF.
4. Brooker, M. (2015). Exponential Backoff And Jitter. AWS Architecture Blog.
5. websockets: a library for building WebSocket servers and clients in Python. Documentation.
6. Reson8 API documentation. docs.reson8.dev.