Coding and terminal agents / Benchmark leaderboard
Android Bench
100 software engineering tasks on Android repositories.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Build/test-based task correctness.
Execution requirements
KVM-capable Linux host, Android tooling and emulators.
Scope and limitations
This tests building/fixing Android software, not using phone apps. The documented local minimum is 32 GB RAM and 120 GB disk.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Newer specialist
- Harbor · android-bench/android-bench ↗100 source tasks · Public · Public detail metadata + family research
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Augment Code: Capability-aligned; adapter/access to qualify
Cognition / Devin: Capability-aligned; adapter/access to qualify
Cursor: Capability-aligned; adapter/access to qualify
GitHub Copilot: Capability-aligned; adapter/access to qualify
Google DeepMind: Capability-aligned; adapter/access to qualify
JetBrains: Capability-aligned; adapter/access to qualify