A completed run still needs evidence.
3 of 10 selected attempts passed the declared primary evaluation. 3 ended with an execution error or timeout. Every selected attempt stays in the report.
online-mind2web / Benchmark deep dive
Compare the outcomes, understand the gaps, and inspect the work behind every score.
3 of 10 selected attempts passed the declared primary evaluation. 3 ended with an execution error or timeout. Every selected attempt stays in the report.
2 task verdicts differ between the rubric and screenshot reviews. The rubric can use captured page text; vision is limited to the recorded frames. Open the task to see what each judgment actually supports.
Dollar cost is recorded for 0 of 10 attempts. Subscription usage and missing provider billing are not converted into an invented per-task price. Latency includes the scope declared in the protocol below.
Scores use the same selected tasks and the report’s declared primary evaluator.
| Model / agent | Success | Execution errors | Mean latency | Cost / task |
|---|---|---|---|---|
| Codex + Playwrightgpt-6-astra | 30.0%3/10 selected tasks | 3 | 182.9s | Not reported |
Pilot coverage: 10/300 source tasks. Small fixed selections do not establish a full-benchmark ranking. Missing grades remain visible.
Execution failure and an incorrect result are separate outcomes.
RuntimeError: Server error '502 Bad Gateway' for url 'http://10.40.192.188:8765/solve' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/502
The work behind the score
Original instructions, observed actions, and the verdict for each attempt.
gpt-6-astra · 335.4s · 15 recorded actions
Search for a new internal M2 Samsung SSD drive between $25 and $200.
Recorded benchmark evaluator · not-configured
Evaluator was not configured
gpt-6-astra · browser-dom-constraints-v1
RuntimeError: Server error '502 Bad Gateway' for url 'http://10.40.192.188:8765/solve' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/502
gpt-6-astra · browser-uniform-eight-frames-v1
RuntimeError: Server error '502 Bad Gateway' for url 'http://10.40.192.188:8765/solve' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/502
Reproducibility
Imported recorded evidence; not independently rerun by the report engine. Scaffold: harbor-playwright-dom-v1 at Harbor 0.21.0; Playwright 1.58.0; Codex CLI 0.154.0. Unknown measurements remain unknown. Fresh headless Chromium per task. Independent DOM rubric primary; at most eight uniformly spaced screenshots including first and last for supplemental vision. Native task-success grading is unconfigured. This is not the official WebJudge protocol or human adjudication. Solver and independent graders use the same model family; dollar costs are unmeasured. Task latency includes browser/solver setup and execution. Per-grader latency and total duration including grading are retained separately in raw outputs. Partial trajectories preserve original execution errors without rescoring.