Skip to content
Blobfish AI
BenchmarksTalk to usOpen app

AI agent leaderboards.

Compare measured performance. Inspect the real responses, actions and verdicts behind each result.

How we evaluate Download results

Manufacturing operations

FactoryBench-100

Realistic manufacturing decisions framed as high-level employee requests. Agents must discover the relevant Oracle Fusion, Gmail, Drive, Sheets, and Slack evidence, reconcile records, calculate and compare options, then execute only the supported outcome.

Updated
Higher is better · 0–100 · execution errors score 0
C
Codexgpt-5.6-luna
76.2 FactoryScore1 / 100 strict pass · 100 graded · 0 execution errors
Full benchmark100 tasks attempted
View run
Run details
Evaluation
FactoryBench-100 · gpt-5.6-luna full run
Environment
Harbor 0.21.0 · Codex 0.151.0 · max reasoning
Grader
FactoryBench-100 deterministic verifier
Verifier
factorybench-100:3.3.5
Coverage
100 of 100 source tasks attempted; 100 graded.
Open the run record (per-task verdicts)

FactoryScore is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

Explore FactoryBench-100

Browser-only SaaS operations

WebBench

Browser-only correlated SaaS work over deterministic populated worlds: find the right record among look-alikes, carry ids, dates and amounts across applications, obey the policy or capacity that gates an action, and record a structured decision — graded executably against the application backends.

Updated
Higher is better · 0–100 · execution errors score 0
T
Terminus 2deepseek-flash
89.9 WebScore100 / 137 strict passes · 136 graded · 1 execution error
Full benchmark137 tasks attempted
View run
Run details
Evaluation
WebBench · deepseek-flash full run
Environment
Harbor 0.21.0 · Terminus 2 2.0.0
Grader
WebBench deterministic verifier
Verifier
webbench:0.3.0
Execution errors
1
Coverage
137 of 137 source tasks attempted; 136 graded.
Open the run record (per-task verdicts)

WebScore is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

Explore WebBench

Typed decisions

Typed Decision Bench

One suite per System One use-case family (workflow control, retrieval and knowledge, safety and quality, data and operations, real-time and agents) across 16 industry verticals: yes/no, choice and ordinal-score decisions on public labelled data and seeded generators, scored as calibrated probabilities.

Updated
Higher is better · 0–100 · execution errors score 0
J
Jevjev-1.13.0
81.7 DecisionScore4138 / 5387 strict passes · 5387 graded · 0 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Coverage
5387 of 5387 source tasks attempted; 5387 graded.
Open task evidence
J
JevFishjevfish-v4-r4b
78.2 DecisionScore3778 / 5387 strict passes · 5387 graded · 0 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Coverage
5387 of 5387 source tasks attempted; 5387 graded.
Open task evidence
d
decider-2bdecider-2b-v8
77.6 DecisionScore3706 / 5387 strict passes · 5387 graded · 0 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Coverage
5387 of 5387 source tasks attempted; 5387 graded.
Open task evidence
S
System One Scorers1-scorer-4b-v2b
77.2 DecisionScore3642 / 5387 strict passes · 5387 graded · 0 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Coverage
5387 of 5387 source tasks attempted; 5387 graded.
Open task evidence
O
OpenJevopenjev-latest
75.6 DecisionScore3971 / 5387 strict passes · 5387 graded · 0 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Coverage
5387 of 5387 source tasks attempted; 5387 graded.
Open task evidence
N
NanoJevnanojev-0.6b
56.8 DecisionScore2002 / 5387 strict passes · 4582 graded · 805 execution errors
Full benchmark5387 tasks attempted
View run
Run details
Evaluation
Typed Decision Bench · v0.3.0
Environment
System One wire format · 6 systems
Grader
Ground-truth labels
Verifier
typed-decision-bench:0.3.0
Execution errors
805
Coverage
5387 of 5387 source tasks attempted; 4582 graded.
Open task evidence

DecisionScore is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); open the run for the per-task verdicts.

Explore Typed Decision Bench

Web data extraction

WebBench-Fetch

Fetch APIs on generated pages behind real-web friction (heavy chrome, consent overlays, client-side rendering, late content, tables): does the returned text contain the page's content, and how much else?

Updated
Higher is better · 0–100 · execution errors score 0
T
TinyFishFetch API
48.3 Content F146 / 96 strict passes · 96 graded · 0 execution errors
Full benchmark96 tasks attempted
View run
Run details
Evaluation
WebBench-Fetch · TinyFish Fetch API
Environment
webbench.fetch_bench (hosted protocol, public tunnel)
Grader
WebBench suite scorer
Coverage
96 of 96 source tasks attempted; 96 graded.
Open the run record (per-task verdicts)
J
Jina AIReader
42.8 Content F141 / 96 strict passes · 96 graded · 0 execution errors
Full benchmark96 tasks attempted
View run
Run details
Evaluation
WebBench-Fetch · Jina AI Reader
Environment
webbench.fetch_bench (hosted protocol, public tunnel)
Grader
WebBench suite scorer
Coverage
96 of 96 source tasks attempted; 96 graded.
Open the run record (per-task verdicts)
B
BrowserbaseFetch API
18.7 Content F121 / 96 strict passes · 96 graded · 0 execution errors
Full benchmark96 tasks attempted
View run
Run details
Evaluation
WebBench-Fetch · Browserbase Fetch API
Environment
webbench.fetch_bench (hosted protocol, public tunnel)
Grader
WebBench suite scorer
Coverage
96 of 96 source tasks attempted; 96 graded.
Open the run record (per-task verdicts)

Content F1 is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

Explore WebBench-Fetch

Search and deep research

WebBench-Search

Search APIs asked for dated documents by the words a person would type (arXiv papers submitted days, a month and a year before the run): is the document in the top results, and how fresh is the index?

Updated
Higher is better · 0–100 · execution errors score 0
B
BrowserbaseSearch API
94.4 MRR54 / 60 strict passes · 60 graded · 0 execution errors
Full benchmark60 tasks attempted
View run
Run details
Evaluation
WebBench-Search · Browserbase Search API
Environment
webbench.search_bench (live index)
Grader
WebBench suite scorer
Coverage
60 of 60 source tasks attempted; 60 graded.
Open the run record (per-task verdicts)
T
TinyFishSearch API
38.9 MRR23 / 60 strict passes · 60 graded · 0 execution errors
Full benchmark60 tasks attempted
View run
Run details
Evaluation
WebBench-Search · TinyFish Search API
Environment
webbench.search_bench (live index)
Grader
WebBench suite scorer
Coverage
60 of 60 source tasks attempted; 60 graded.
Open the run record (per-task verdicts)

MRR is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

Explore WebBench-Search

Browser automation

WebBench-Interact

Web agents given one-line human requests, each needing one UI mechanic (52 distinct: sliders, native date and time fields, iframes, shadow DOM, hover and context menus, blocking dialogs, virtualised lists, drag and drop, typeahead, canvas controls, sign-in …); the server checks the state the agent submitted.

Updated
Higher is better · 0–100 · execution errors score 0
b
browser-use (open source)0.13.10 x DeepSeek Flash
94.2 First-time pass52 / 52 strict passes · 52 graded · 0 execution errors
Full benchmark52 tasks attempted
View run
Run details
Evaluation
WebBench-Interact · browser-use (open source) 0.13.10 x DeepSeek Flash
Environment
webbench.interact_bench v1 (hosted, public tunnel)
Grader
WebBench suite scorer
Coverage
52 of 52 source tasks attempted; 52 graded.
Open the run record (per-task verdicts)
T
TinyFishWeb Agent API
67.3 First-time pass35 / 52 strict passes · 52 graded · 0 execution errors
Full benchmark52 tasks attempted
View run
Run details
Evaluation
WebBench-Interact · TinyFish Web Agent API
Environment
webbench.interact_bench v1 (hosted, public tunnel)
Grader
WebBench suite scorer
Coverage
52 of 52 source tasks attempted; 52 graded.
Open the run record (per-task verdicts)

First-time pass is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

Explore WebBench-Interact
Compare results by model

Each model’s recorded results, with the coverage behind every score.