Scientific and ML agents / Benchmark leaderboard
RE-Bench
Autonomous AI research and engineering tasks with varying time budgets.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Task-specific improvement normalized by the published methodology.
Execution requirements
Research coding environment and fixed compute/time limits.
Scope and limitations
Small expert-designed suite; repeated exposure and resource differences can dominate comparisons.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established specialist
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Anthropic: Capability-aligned; adapter/access to qualify
Google DeepMind: Capability-aligned; adapter/access to qualify
NVIDIA: Research / simulator adaptation
OpenAI: Capability-aligned; adapter/access to qualify
Sakana AI: Capability-aligned; adapter/access to qualify