Healthcare AI / Benchmark leaderboard
MedAgentBench
300 tasks requiring action in a virtual FHIR-compatible EHR.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Task completion against environment state.
Execution requirements
FHIR/EHR action adapter and benchmark environment.
Scope and limitations
Only relevant to an action-capable product; a scribe without EHR tool access cannot complete the full suite.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established newer specialist
- Harbor · stanford/medagentbench ↗300 source tasks · Public · Public detail metadata + family research
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Abridge: Product-specific adapter / qualified access
Ambience: Product-specific adapter / qualified access
Google DeepMind: Capability-aligned; adapter/access to qualify
Hippocratic AI: Product-specific adapter / qualified access
Nabla: Product-specific adapter / qualified access
Suki: Product-specific adapter / qualified access