Coding and terminal agents / Benchmark leaderboard
LiveCodeBench
Competition-style coding, with date-bounded problem releases.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Pass@1 / pass@k on the declared task window.
Execution requirements
Code generation and isolated test execution.
Scope and limitations
Select problems newer than the evaluated model training cutoff where possible. Harbor has a 100-task port, not the entire evolving dataset.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established and refreshed
- Dataset / catalog ↗
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Alibaba Qwen: Capability-aligned; adapter/access to qualify
Anthropic: Capability-aligned; adapter/access to qualify
Cognition / Devin: Capability-aligned; adapter/access to qualify
Cursor: Capability-aligned; adapter/access to qualify
DeepSeek: Capability-aligned; adapter/access to qualify
Factory: Capability-aligned; adapter/access to qualify
GitHub Copilot: Capability-aligned; adapter/access to qualify
JetBrains: Capability-aligned; adapter/access to qualify
Mistral: Capability-aligned; adapter/access to qualify
OpenAI: Capability-aligned; adapter/access to qualify