Coding and terminal agents / Benchmark leaderboard
Terminal-Bench 4.0
Current continuous Terminal-Bench release with task and resource revisions; catalog snapshot lists 66 tasks.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Verifier-defined success under the versioned runtime budget.
Execution requirements
Harbor terminal agent; some tasks need GPUs or multiple containers.
Scope and limitations
The release sets an eight-hour agent timeout. Use the exact v4.0.0 task manifest; a shorter-budget run needs its own label. 3.0 and 2.1 remain separate.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Current release; higher run cost
- Harbor · terminal-bench/terminal-bench ↗66 source tasks · Public · Public detail metadata + family research
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
All Hands / OpenHands: Capability-aligned; adapter/access to qualify
Anthropic: Capability-aligned; adapter/access to qualify
Augment Code: Capability-aligned; adapter/access to qualify
Cline: Capability-aligned; adapter/access to qualify
Cognition / Devin: Capability-aligned; adapter/access to qualify
Cursor: Capability-aligned; adapter/access to qualify
Factory: Capability-aligned; adapter/access to qualify
GitHub Copilot: Capability-aligned; adapter/access to qualify
Google DeepMind: Capability-aligned; adapter/access to qualify
JetBrains: Capability-aligned; adapter/access to qualify
OpenAI: Capability-aligned; adapter/access to qualify
Replit: Capability-aligned; adapter/access to qualify