Skip to content

Browser-only SaaS operations

WebBench

Browser-only correlated SaaS work over deterministic populated worlds: find the right record among look-alikes, carry ids, dates and amounts across applications, obey the policy or capacity that gates an action, and record a structured decision — graded executably against the application backends.

Browser-only SaaS operations

Model performance

1 recorded configuration · 137 source tasks

Updated
Higher is better · 0–100 · execution errors score 0
T
Terminus 2deepseek-flash
89.9 WebScore100 / 137 strict passes · 136 graded · 1 execution error
Full benchmark137 tasks attempted
View run
Run details
Evaluation
WebBench · deepseek-flash full run
Environment
Harbor 0.21.0 · Terminus 2 2.0.0
Grader
WebBench deterministic verifier
Verifier
webbench:0.3.0
Execution errors
1
Coverage
137 of 137 source tasks attempted; 136 graded.
Open the run record (per-task verdicts)

WebScore is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

About WebBench

What it measures

Executable WebScore verification tree per task: gates for containment, evidence before the first commit, no canary or injected write, and abstention; weighted leaves for evidence reads, state rows, answer fields, cross-application consistency and read-backs, graded from the pulled world database and the server-side page trace. No LLM judge; exact call order is not graded.

Scope & limitations

This Blobfish suite has its own task and environment definitions. Upstream scores and reference controls are not company performance scores.

Evaluation environment

The exact 0.3.0 release contains 137 tasks from 13 calibrated templates over 3 populated families (deskops, itsmdesk, workplace), each served as 10–12 server-rendered web applications over one SQLite world. The agent's only surface is the browser CLI; MCP, REST and any tool CLI answer 404. Every task shipped with its nine build-time controls, passed the Docker oracle gate 137/137 and a registry round-trip 5/5. Templates were calibrated on DeepSeek V4.1 Flash; those are calibration measurements, not leaderboard results.

Source

Original benchmark

Dataset & task catalog ↗