MedASR Bench · Clinical Scribe Program

The Vietnamese Medical ASR Benchmark

One harness measuring how speech recognition survives real Vietnamese medical speech — and the English clinical terms code-switched through it — across a growing registry of datasets. Six metrics, bootstrap 95% CIs, transparent provenance on every number.

Datasets: 1 live · 2 onboarding
Anchor set: ViMedCSS v1
Normalizer: v0.1
Harness: medasr-bench v0.4
Systems on anchor: 17

Datasets

Anchor dataset snapshot — ViMedCSS v1

Dataset card

audio: 34.57h
utterances: 16,576
CS medical terms: 889
clip length: 2–21s

On ViMedCSS, Vietnamese-specialized models win the overall word error rate but lose the code-switched spans; multilingual models invert the trade — adaptation closes both.

Leaderboard — ViMedCSS test split

Track

Class

1Gemini 3.5 Flash (audio)
Google
API snapshot
13.84WER29.38CS-WER
2Zipformer-30M RNNT (VLSP 2025)
hynt · 30M
Platform-verified
19.81WER74.55CS-WER
3Qwen3-ASR-0.6B
Alibaba · 600M
Platform-verified
21.29WER68.49CS-WER
4PhoWhisper-small + Attention-Guide
ViMedCSS paper · 244M
Paper-imported
23.67WER19.50CS-WER
5PhoWhisper-small + LoRA
ViMedCSS paper · 244M
Paper-imported
27.13WER30.26CS-WER
6VietASR (Zipformer 68M)
SJTU / zzasdf · 68M
Paper-imported
27.56WER58.38CS-WER
7PhoWhisper-large
VinAI · 1.55B
Paper-imported
31.24WER55.05CS-WER
8Deepgram Nova-3
Deepgram
API snapshot
31.53WER70.75CS-WER
9Whisper-large-v3
OpenAI · 1.55B
Paper-imported
34.47WER46.69CS-WER
10PhoWhisper-small
VinAI · 244M
Paper-imported
36.31WER
11wav2vec2-base-vi
— · 95M
Paper-imported
44.74WER
12Whisper-small
OpenAI · 244M
Paper-imported
50.03WER
13MMS-1B
Meta · 1B
Paper-imported
58.30WER

Unscored on Test — awaiting runs or imports

—Gemini 2.5 Pro (audio)
Google
Planned
—
—gpt-4o-transcribe
OpenAI
Planned
—
—ElevenLabs Scribe v2
ElevenLabs
Planned
—
—FPT.AI STT
FPT Smart Cloud
Planned
—

Model leaderboard on the ViMedCSS Test split, sorted by WER
#	Model	Params	Track	License							Provenance
1	Gemini 3.5 Flash (audio) Google	—	A	API	13.84	29.38	13.26	12.22	71.380.8 fz	—	API snapshot
2	Zipformer-30M RNNT (VLSP 2025) hynt	30M	A	CC-BY-NC-ND-4.0	19.81	74.55	17.78	15.92	27.336.5 fz	48.1×	Platform-verified
3	Qwen3-ASR-0.6B Alibaba	600M	A	Apache-2.0	21.29	68.49	19.54	16.11	32.342.3 fz	5.0×	Platform-verified
4	PhoWhisper-small + Attention-Guide ViMedCSS paper	244M	C	BSD-3-Clause	23.67	19.50					Paper-imported
5	PhoWhisper-small + LoRA ViMedCSS paper	244M	C	BSD-3-Clause	27.13	30.26					Paper-imported
6	VietASR (Zipformer 68M) SJTU / zzasdf	68M	A	Apache-2.0	27.56	58.38					Paper-imported
7	PhoWhisper-large VinAI	1.55B	A	BSD-3-Clause	31.24	55.05					Paper-imported
8	Deepgram Nova-3 Deepgram	—	A	API	31.53	70.75	30.07	28.45	30.038.8 fz	—	API snapshot
9	Whisper-large-v3 OpenAI	1.55B	A	Apache-2.0	34.47	46.69					Paper-imported
10	PhoWhisper-small VinAI	244M	A	BSD-3-Clause	36.31						Paper-imported
11	wav2vec2-base-vi —	95M	A	Open	44.74						Paper-imported
12	Whisper-small OpenAI	244M	A	Apache-2.0	50.03						Paper-imported
13	MMS-1B Meta	1B	A	CC-BY-NC-4.0	58.30						Paper-imported
Unscored on Test · WER — awaiting runs or imports
—	Gemini 2.5 Pro (audio) Google	—	A	API	—	—	—	—	—	—	Planned
—	gpt-4o-transcribe OpenAI	—	A	API	—	—	—	—	—	—	Planned
—	ElevenLabs Scribe v2 ElevenLabs	—	B	API	—	—	—	—	—	—	Planned
—	FPT.AI STT FPT Smart Cloud	—	A	API	—	—	—	—	—	—	Planned

17 / 17 models · Test split · sorted by WER

[1]All WER-family numbers were computed under text normalizer v0.1. WER is only comparable within a normalizer version; the version is recorded per run.
[2]Paper-imported rows (dashed chips) reproduce numbers from arXiv:2602.12911 as published — not yet re-verified on this platform.
[3]Contamination flag: ViMedCSS labels were seeded by Gemini 2.5 Pro before human verification (κ = 0.65), so Gemini-family models carry a permanent contamination flag. With public labels, provenance declarations are honor-system for now; a private canary subset is on the roadmap.
[4]Domain caveat: ViMedCSS is lecture-style YouTube audio, not noisy two-speaker clinic conversation. Scores here do not certify the production deployment gate (WER < 15% on clinic speech).
[5]Bars under WER and CS-WER place the value on a fixed 15–65 scale; whiskers appear when bootstrap 95% CIs are available — platform runs record them, paper imports rarely publish them.

Leaderboard — ViMedCSS test split

Track

Class

1Gemini 3.5 Flash (audio)
Google
API snapshot
13.84WER29.38CS-WER
2Zipformer-30M RNNT (VLSP 2025)
hynt · 30M
Platform-verified
19.81WER74.55CS-WER
3Qwen3-ASR-0.6B
Alibaba · 600M
Platform-verified
21.29WER68.49CS-WER
4PhoWhisper-small + Attention-Guide
ViMedCSS paper · 244M
Paper-imported
23.67WER19.50CS-WER
5PhoWhisper-small + LoRA
ViMedCSS paper · 244M
Paper-imported
27.13WER30.26CS-WER
6VietASR (Zipformer 68M)
SJTU / zzasdf · 68M
Paper-imported
27.56WER58.38CS-WER
7PhoWhisper-large
VinAI · 1.55B
Paper-imported
31.24WER55.05CS-WER
8Deepgram Nova-3
Deepgram
API snapshot
31.53WER70.75CS-WER
9Whisper-large-v3
OpenAI · 1.55B
Paper-imported
34.47WER46.69CS-WER
10PhoWhisper-small
VinAI · 244M
Paper-imported
36.31WER
11wav2vec2-base-vi
— · 95M
Paper-imported
44.74WER
12Whisper-small
OpenAI · 244M
Paper-imported
50.03WER
13MMS-1B
Meta · 1B
Paper-imported
58.30WER

Unscored on Test — awaiting runs or imports

—Gemini 2.5 Pro (audio)
Google
Planned
—
—gpt-4o-transcribe
OpenAI
Planned
—
—ElevenLabs Scribe v2
ElevenLabs
Planned
—
—FPT.AI STT
FPT Smart Cloud
Planned
—

Model leaderboard on the ViMedCSS Test split, sorted by WER
#	Model	Params	Track	License							Provenance
1	Gemini 3.5 Flash (audio) Google	—	A	API	13.84	29.38	13.26	12.22	71.380.8 fz	—	API snapshot
2	Zipformer-30M RNNT (VLSP 2025) hynt	30M	A	CC-BY-NC-ND-4.0	19.81	74.55	17.78	15.92	27.336.5 fz	48.1×	Platform-verified
3	Qwen3-ASR-0.6B Alibaba	600M	A	Apache-2.0	21.29	68.49	19.54	16.11	32.342.3 fz	5.0×	Platform-verified
4	PhoWhisper-small + Attention-Guide ViMedCSS paper	244M	C	BSD-3-Clause	23.67	19.50					Paper-imported
5	PhoWhisper-small + LoRA ViMedCSS paper	244M	C	BSD-3-Clause	27.13	30.26					Paper-imported
6	VietASR (Zipformer 68M) SJTU / zzasdf	68M	A	Apache-2.0	27.56	58.38					Paper-imported
7	PhoWhisper-large VinAI	1.55B	A	BSD-3-Clause	31.24	55.05					Paper-imported
8	Deepgram Nova-3 Deepgram	—	A	API	31.53	70.75	30.07	28.45	30.038.8 fz	—	API snapshot
9	Whisper-large-v3 OpenAI	1.55B	A	Apache-2.0	34.47	46.69					Paper-imported
10	PhoWhisper-small VinAI	244M	A	BSD-3-Clause	36.31						Paper-imported
11	wav2vec2-base-vi —	95M	A	Open	44.74						Paper-imported
12	Whisper-small OpenAI	244M	A	Apache-2.0	50.03						Paper-imported
13	MMS-1B Meta	1B	A	CC-BY-NC-4.0	58.30						Paper-imported
Unscored on Test · WER — awaiting runs or imports
—	Gemini 2.5 Pro (audio) Google	—	A	API	—	—	—	—	—	—	Planned
—	gpt-4o-transcribe OpenAI	—	A	API	—	—	—	—	—	—	Planned
—	ElevenLabs Scribe v2 ElevenLabs	—	B	API	—	—	—	—	—	—	Planned
—	FPT.AI STT FPT Smart Cloud	—	A	API	—	—	—	—	—	—	Planned

17 / 17 models · Test split · sorted by WER