Speech recognition / Benchmark leaderboard
Vaani Benchmark v1.0
Speech from diverse Indian districts and speakers with multiple reference transcripts.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Benchmark-specific multilingual transcription scores.
Execution requirements
ASR with supported Indian languages.
Scope and limitations
Gated access and language-specific normalization; supplement rather than replace global speech coverage.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Newer regional specialist
- Hugging Face · ARTPARK-IISc/Vaani-Benchmark-V1.0 ↗Gated · Selected Indian-language ASR specialist; gated.
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
AssemblyAI: Direct ASR API
Cartesia: TTS API; full cloning protocol conditional
Deepgram: Capability-aligned; adapter/access to qualify
Gladia: Direct ASR API
Google DeepMind: Capability-aligned; adapter/access to qualify
Meta: Capability-aligned; adapter/access to qualify
Rev: Direct ASR API
Soniox: Direct ASR API
Speechmatics: Direct ASR API