On-premise and VPC deployment run a speech recognition model inside infrastructure the customer controls, so audio never leaves the customer's network. On-premise means the customer's own servers; VPC means an isolated network in the customer's cloud account. Both remove the vendor from the data path, which settles most residency and retention questions, at the price of the customer taking on GPU capacity, operations and upgrades.
Key takeaways
- Self-hosting moves the model to the data rather than the data to the model. The vendor supplies software and weights; the customer runs them.
- It is the strongest answer to residency, retention and foreign-access concerns, because the vendor never receives audio.
- The customer inherits operational work: GPU procurement, capacity planning for peak concurrency, monitoring and version upgrades.
- Throughput per GPU matters far more when you pay for idle hardware yourself. Inference engine efficiency becomes a direct cost.
- Agree up front how updates, licence checks, telemetry and support access work, especially for air-gapped sites.
Deployment models#
Speech recognition can be consumed in several ways, which differ in where the model runs and who operates it.
| Model | Where inference runs | Who operates | Audio reaches vendor? |
|---|---|---|---|
| Multi-tenant SaaS | Vendor infrastructure, shared | Vendor | Yes |
| Dedicated SaaS | Vendor infrastructure, isolated | Vendor | Yes |
| VPC | Customer's cloud account | Customer, sometimes vendor-assisted | No |
| On-premise | Customer's data centre | Customer | No |
| Air-gapped | Isolated customer network | Customer | No, and no network path exists |
A related option is on-device recognition, where a smaller model runs on a phone, car or embedded board. That trades accuracy for locality in a way that server-class self-hosting does not have to.
Why organisations self-host#
- Data never leaves the perimeter. For hospitals, courts, defence and some financial institutions, policy forbids sending recordings to a third party regardless of contract terms. Self-hosting removes the question.
- Jurisdiction. Self-hosting takes the speech vendor, and any foreign disclosure laws that apply to it, out of the data path. In a VPC, your cloud provider's jurisdiction still applies, so on-premise is the stricter option. See EU data residency and sovereign AI.
- Network locality. For streaming and voice agents, placing the recogniser next to the telephony platform or agent logic removes an internet round trip from the latency budget.
- Predictable cost at high volume. Organisations transcribing very large volumes may prefer owned GPU capacity to per-minute pricing, provided utilisation stays high.
- Existing cloud commitments. Deploying into a VPC lets spend count against committed cloud contracts and keeps networking under existing controls.
What the customer takes on#
GPU capacity#
Speech models run on GPUs, and the number required depends on peak concurrent streams for real-time use and on total audio hours for batch. A batch workload can queue and run at full utilisation. A real-time workload must be sized for the busiest hour, and capacity sits idle the rest of the time. The real-time factor of the engine, and how many streams it can serve per GPU without latency degrading, decides the hardware bill. Concurrency and scaling covers the sizing method.
Upgrades and model versions#
SaaS customers receive model improvements automatically. Self-hosted customers must pull new container images and weights, test them, and roll them out. That is also an advantage: nothing changes without your approval, which matters in regulated settings where a model change requires revalidation. Keep a test set from your own domain and run it on every new version before promotion.
Observability without content#
The vendor cannot see your logs, so support depends on what you choose to share. Plan for metrics that contain no content, such as request counts, latency percentiles, GPU memory and error codes, which can be sent to the vendor or reviewed together without exposing audio or transcripts.
Network and licensing#
Decide early whether the deployment may make any outbound connections. Some vendors require a licence check or send usage telemetry for billing. Air-gapped sites need an offline licensing scheme and a process for importing signed images through a controlled transfer.
Shared responsibility#
Self-hosting changes who is accountable for what. The vendor remains responsible for the software: the quality of the model, the security of its containers and dependencies, and timely patches. The customer becomes responsible for the environment: host hardening, network segmentation, identity and access, key management, backups and incident response. Under GDPR, if the vendor never receives personal data it may not be a processor for the inference at all, though it may become one again if its staff are given remote access for support. Write down which parties may access the environment and how.
How Reson8 approaches this#
Reson8's hosted service runs on EU GPUs. On-premise and VPC deployment are available on enterprise plans, under the Custom tier that also carries an SLA. Resonant-1 is served by Violin, Reson8's own inference engine for speech, which uses fused kernels, CUDA graphs and decoding scheduling to raise throughput; that throughput (an RTFx of 1187 on the Open ASR Leaderboard) is what determines how much hardware a self-hosted deployment needs. Details are in the documentation.
Common questions#
Is a dedicated SaaS instance the same as a VPC deployment?
No. A dedicated instance is isolated from other customers but still runs on the vendor's infrastructure, so audio reaches the vendor. A VPC deployment runs in your own cloud account.
Does self-hosting reduce accuracy?
It should not, if the same model and inference engine are deployed. Accuracy differences usually come from older versions left in place or from different hardware settings, so compare on your own test set.
Do I still need a DPA for on-premise use?
If the vendor never processes your personal data, Article 28 may not apply to inference. It applies again if vendor staff access the environment or receive audio for support, so many customers sign one anyway.
Can domain adaptation work self-hosted?
That depends on the vendor. Where adaptation is text-based and applied at runtime, it does not need training infrastructure, which makes it easier to offer in a self-hosted setup. Check the vendor's documentation.
References#
- Mell, P., & Grance, T. (2011). The NIST Definition of Cloud Computing. NIST Special Publication 800-145. NIST.
- Rose, S., Borchert, O., Mitchell, S., & Connelly, S. (2020). Zero Trust Architecture. NIST Special Publication 800-207. NIST.
- Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 28 and 32. EUR-Lex.
- NVIDIA. CUDA C++ Programming Guide: CUDA Graphs. NVIDIA.
- Srivastav, V. et al. Open ASR Leaderboard. Hugging Face.