Software engineering
RepoBench-500
Expert-verified GitHub issues paired with a service, repository, pull-request, incident, ADR, and standup world.
About RepoBench-500
What it measures
Expert-verified, solvable real-world GitHub issues scored by repository tests.
Scope & limitations
This Blobfish suite has its own task and environment definitions. Upstream scores and reference controls are not company performance scores.
Evaluation environment
The upstream task environments use repository-specific Docker images and test contracts.