Model profile · Commercial API (intl)

Gemini 2.5 Pro (audio)

Planned

Google—APITrack A — Zero-shot

Seeded this dataset labels — permanently contamination-flagged

Contamination — this model’s family seeded the ViMedCSS labels (Gemini 2.5 Pro, then human-verified at κ 0.65), so its scores carry a permanent contamination flag.

Results onViMedCSSVietnamese Medical Code-Switching Speech34.57 h · 16,576 utts · 889 CS terms

Dataset profile

Run manifest

normalizer: v0.1
harness: medasr-bench v0.4
decode_config: —
checkpoint: —
run_date: —
source: Planned
dataset: ViMedCSS · test 3.38 h / hard 1.38 h
splits: —

WER is comparable only within a normalizer version · bootstrap 95% CIs on headline metrics.

Roadmap — evaluation planned

This model is queued on the evaluation roadmap and has not been run yet. Commercial APIs are benchmarked as date-stamped snapshots, with quarterly drift re-runs planned.

awaiting first platform run

No trace exists for this model on ViMedCSSyet — measurements, Test → Hard generalization and the per-topic breakdown all land with its first run on Test + Hard.

Queue position 1 of 4 unmeasured systems

WER
CS-WER
N-WER
CER
MTR
RTFx
Test → Hard Δ
per-topic WER