# WebVoyager evaluation

Report: `WebVoyager · 15-task Browser Use pilot · agent evaluation`
Dataset version: `5a7896738c10bfb8b9edccce6bb0e0411f8ae569:69b19fd86c23f1a500244a3724e039aa7ca6a1223d03e11eb10e308d4f11c488`
Primary evaluator: `rubric` — Primary: rubric; declared grading contract 82db14ea2dc5d3458d75a485c7de1eb4eefad97fca0871ea20146a4c8f0cb006
Environment: Browser Use Cloud v4; fresh hosted browser per task; US proxy
Budget: 300s/task; $1.0 agent cap/task; one accepted run per task

Selected tasks per agent: 15. Official total: 643.
These results cover the selected tasks and the recorded execution budgets only. Native, rubric and vision verdicts are separate. Unavailable graders are not passes; a pass rate is reported only with complete grading coverage. No retries are silently substituted.

Protocol: Imported recorded evidence; not independently rerun by the report engine. Scaffold: browser-use-cloud-v4 at SDK 3.11.3. Unknown measurements remain unknown. 15/643 tasks; one per website, seed 20260915. Separate Codex AI evidence review, not official GPT-4V grading or human adjudication; review model version was not retained. Unverified requirements receive no success credit under the recorded rubric. A later, separate Astra vision review runs on GKE using only the final screenshot and answer. Its restricted evidence scope is supplemental and does not replace the original rubric primary. Vision latency and subscription token usage are recorded separately; no dollar cost is inferred. Provider trajectory judgments are retained separately. Latency measures create-to-terminal agent time. Cost is final provider-reported run cost including asynchronous judge updates; browser hosting is separate. Original fixed dates were preserved.

## Browser Use v4

Model: `gpt-5.6-luna`; run: `webvoyager-browser-use-20260915-vision-20260916:browser-use-v4`; source: imported; 2026-09-16T00:31:33.909105+00:00 → 2026-09-16T00:43:45.396405+00:00.

| Site / category | Tasks | Completed | Primary passes | Native passes / graded | Rubric passes / graded | Vision passes / graded | Mean latency |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| Allrecipes | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 41.48s |
| Amazon | 1 | 1 | 0 (0.0%) | — | 0 / 1 (0.0%) | 0 / 1 (0.0%) | 102.31s |
| Apple | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 92.15s |
| ArXiv | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 1 / 1 (100.0%) | 101.12s |
| BBC News | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 182.26s |
| Booking | 1 | 1 | 0 (0.0%) | — | 0 / 1 (0.0%) | 0 / 1 (0.0%) | 202.18s |
| Cambridge Dictionary | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 1 / 1 (100.0%) | 30.51s |
| Coursera | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 121.99s |
| ESPN | 1 | 1 | 0 (0.0%) | — | 0 / 1 (0.0%) | 0 / 1 (0.0%) | 60.69s |
| GitHub | 1 | 1 | 0 (0.0%) | — | 0 / 1 (0.0%) | 0 / 1 (0.0%) | 111.19s |
| Google Flights | 1 | 1 | 0 (0.0%) | — | 0 / 1 (0.0%) | 0 / 1 (0.0%) | 164.06s |
| Google Map | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 1 / 1 (100.0%) | 50.52s |
| Google Search | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 166.81s |
| Huggingface | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 96.58s |
| Wolfram Alpha | 1 | 1 | 1 (100.0%) | — | 1 / 1 (100.0%) | 0 / 1 (0.0%) | 244.21s |
| Overall | 15 | 15 | 10 (66.7%) | — | 10 / 15 (66.7%) | 3 / 15 (20.0%) | 117.87s |

P95 latency: 244.21s (15/15 timed). Total recorded model cost: $0.5882 ($0.0392 per task).
Execution errors or timeouts: 0. Completed but ungraded: 0. Not attempted: 0.

Notes: Imported recorded evidence; not independently rerun by the report engine. Scaffold: browser-use-cloud-v4 at SDK 3.11.3. Unknown measurements remain unknown. 15/643 tasks; one per website, seed 20260915. Separate Codex AI evidence review, not official GPT-4V grading or human adjudication; review model version was not retained. Unverified requirements receive no success credit under the recorded rubric. A later, separate Astra vision review runs on GKE using only the final screenshot and answer. Its restricted evidence scope is supplemental and does not replace the original rubric primary. Vision latency and subscription token usage are recorded separately; no dollar cost is inferred. Provider trajectory judgments are retained separately. Latency measures create-to-terminal agent time. Cost is final provider-reported run cost including asynchronous judge updates; browser hosting is separate. Original fixed dates were preserved.