Medical speech recognition transcribes clinical speech: dictated letters and reports, and increasingly whole consultations. The vocabulary is dense with drug names, anatomy, abbreviations and dosages, and a single wrong word can change a treatment. Aggregate word error rate is a poor guide here; what matters is how often the system gets the clinically important words wrong, and whether the workflow catches those errors before a note is signed.
Key takeaways
- Three workflows dominate: front-end dictation (the clinician corrects), back-end dictation (a transcriptionist corrects), and ambient documentation of consultations.
- The dangerous errors are few and specific: drug names, doses, laterality, negation and numbers. Measure them separately from overall WER.
- General models miss specialist terms. Domain adaptation with drug formularies, specialty glossaries and past reports is usually the largest single improvement.
- Health data is a special category under GDPR Article 9. Processing needs a condition under Article 9(2) as well as a lawful basis, plus a processor agreement with the vendor.
- Plain transcription is generally not a medical device, but software that interprets clinical content may fall under the EU Medical Device Regulation.
Where medical ASR is used#
Dictation is the oldest use. Radiologists dictate reports, specialists dictate referral and discharge letters, pathologists dictate findings. Radiology was an early adopter because reports are long, structured and produced in high volume.
- Front-end: text appears as the clinician speaks, often into a report template in the radiology or electronic health record (EHR) system. The clinician corrects and signs. Fast turnaround, but the clinician carries the editing burden.
- Back-end: the clinician dictates and moves on. The recording is transcribed in batch, a medical transcriptionist edits the draft, and the clinician reviews and signs. Turnaround is slower; the editing burden moves to a specialist.
- Ambient: a device records the consultation and a language model drafts the note. This is covered in ambient clinical documentation.
Smaller uses include nursing handovers, operating-theatre notes, voice commands to navigate an EHR, and transcription of telephone triage lines, where audio arrives at 8 kHz (see telephony transcription).
Why clinical vocabulary is hard#
Clinical language combines a long tail of rare terms with high stakes. A general model has seen "metformin" often enough; it has seen "pembrolizumab", "ticagrelor" or "Klatskin tumour" rarely, if at all. When a model does not know a word, it produces the nearest common words instead: ticagrelor becomes tick a grill or.
| Error type | Said | Heard | Why it matters |
|---|---|---|---|
| Sound-alike drug | hydroxyzine | hydralazine | An antihistamine becomes an antihypertensive |
| Dose | fifteen milligrams | fifty milligrams | More than three times the intended dose |
| Decimal | zero point five | 5 | Tenfold error |
| Laterality | left kidney | right kidney | Wrong-site risk |
| Negation | no evidence of metastasis | evidence of metastasis | Opposite finding |
| Split term | metformin | met for men | Unreadable; easy to catch, costly to fix |
The Institute for Safe Medication Practices maintains a published list of confused drug name pairs, and those pairs are a useful starting point for a targeted test. Beyond drugs, clinicians use abbreviations that collide with ordinary words, eponyms (Crohn, Hashimoto, Parkinson), and in many European countries a mix of the local language, Latin and English. A Dutch cardiologist might say "patiënt met atriumfibrilleren, gestart met apixaban, CHA₂DS₂-VASc van drie" in one breath.
Measuring accuracy in clinical settings#
Word error rate weights "the" and "warfarin" equally, so it underrates the difference between systems on the words that matter. Clinical evaluations typically add:
- Medical term accuracy: recall and precision on a list of domain terms, a specialised form of entity and keyword accuracy.
- Critical error rate: errors reviewed by clinicians and labelled for whether they could affect care.
- Residual error after review: errors still present in the signed document. This measures the whole workflow, not the model alone.
Published studies support measuring the whole workflow. Hodgson and Coiera's 2016 systematic review found that speech recognition could shorten turnaround times but was associated with more errors than conventional transcription in several studies. Zhou and colleagues (2018) analysed dictated notes at two US health systems and found errors in the raw ASR drafts, fewer after transcriptionist editing, and some still present in the notes clinicians signed. Editing catches most mistakes, but not all.
Regulatory context in the EU#
GDPR#
Data concerning health is a special category under Article 9 of the GDPR, and dictations and consultation recordings almost always contain it. Processing requires both a lawful basis under Article 6 and a condition under Article 9(2); for care delivery this is usually Article 9(2)(h), processing necessary for medical diagnosis or the provision of health care, under the obligation of professional secrecy described in Article 9(3). A transcription vendor normally acts as a processor, so an Article 28 data processing agreement is required. Large-scale processing of health data will usually require a data protection impact assessment under Article 35. See GDPR and voice data.
Medical Device Regulation#
Under Regulation (EU) 2017/745 (the MDR), software is a medical device when its manufacturer intends it for a medical purpose such as diagnosis, monitoring or treatment. The Medical Device Coordination Group's guidance MDCG 2019-11 explains how to qualify software. Software that only stores, transfers or displays data, or performs simple search, is generally not a device. A transcription engine that turns dictation into text falls on that side. Features that interpret clinical content, such as suggesting a diagnosis or flagging a drug interaction, can change the analysis. The intended purpose, as stated by the manufacturer, decides.
EU AI Act#
Under Regulation (EU) 2024/1689, AI systems that are medical devices, or safety components of them, requiring third-party conformity assessment under the MDR are treated as high-risk. A general transcription service is not a medical device by default, but a vendor or hospital building a clinical product on top of it should check. See the EU AI Act and speech recognition.
Deployment considerations#
- Microphones: dedicated dictation microphones and headsets give far better audio than a laptop microphone in a shared office. Test with the hardware clinicians will actually use.
- Integration: text has to land in the right field of the EHR or reporting system. HL7 FHIR is the common standard for exchanging clinical documents, though many dictation integrations still work through the reporting application directly.
- Formatting: clinicians dictate "new paragraph", "full stop" and section headings. Doses should render as "0.5 mg", not "zero point five milligrams" (see inverse text normalisation).
- Residency and retention: many hospitals require processing inside the EU or on premises, and deletion of audio once transcribed. See EU data residency and zero data retention.
How Reson8 approaches this#
Reson8's founders previously built large-scale medical transcription platforms used in daily clinical workflows, and healthcare is one of the industries Reson8 serves. Its domain adaptation is text-only: a hospital or vendor supplies formularies, glossaries or example reports through the API, with no audio collection or retraining, and the adaptation takes effect in under 60 seconds. It supports 50,000+ medical terms with no latency impact.
Processing runs on EU GPUs with zero audio retention and no training on customer data, under a GDPR Article 28 DPA. On-premise and VPC deployment are available on enterprise plans. Resonant-1 covers nine European languages, so a clinic working in, say, Greek or Czech would need another provider.
Common questions#
Is medical speech recognition accurate enough to skip review?
No. Even strong systems make errors on rare terms, and fluent errors are hard to spot. Clinical documents should be reviewed before signing, and the organisation should track errors that survive review.
Does a medical ASR model need to be trained on medical audio?
Not necessarily. Much of the gap between general and medical accuracy is vocabulary, and context biasing or text-only adaptation can close a large part of it without new audio. Acoustic conditions, such as a noisy ward, are a separate problem.
Is a transcription tool a medical device?
Usually not, if it only converts speech to text. It can become one if its intended purpose includes diagnosis, monitoring or treatment. MDCG 2019-11 is the guidance to read, and the manufacturer's stated intended purpose is decisive.
How do clinicians dictate punctuation?
Traditionally by speaking commands such as "comma" and "new paragraph". Modern models punctuate automatically, but many clinicians keep the habit, so the system should accept both.
References#
- Hodgson, T., & Coiera, E. (2016). Risks and benefits of speech recognition for clinical documentation: a systematic review. Journal of the American Medical Informatics Association, 23(e1).
- Zhou, L. et al. (2018). Analysis of Errors in Dictated Clinical Documents Assisted by Speech Recognition Software and Professional Transcriptionists. JAMA Network Open, 1(3).
- Institute for Safe Medication Practices. List of Confused Drug Names. ISMP.
- Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 9, 28 and 35. EUR-Lex.
- Regulation (EU) 2017/745 on medical devices. EUR-Lex.
- Medical Device Coordination Group (2019). MDCG 2019-11: Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 and Regulation (EU) 2017/746. European Commission.
- HL7 International. FHIR specification. hl7.org.