Skip to content

Coding and terminal agents

SWE-bench Pro

Long-horizon repository-level software tasks; public Harbor snapshot has 731 tasks.

About SWE-bench Pro

What it measures

Resolved-task rate.

Scope & limitations

Use the public split identity; do not treat Pro, Verified and private commercial evaluations as interchangeable.

Evaluation environment

Repository agent and Pro-specific environments/tests.

Source

Original benchmark

Dataset & task catalog ↗