Skip to content
Blobfish AI
BenchmarksTalk to usOpen app

Manufacturing operations

FactoryBench-100

Realistic manufacturing decisions framed as high-level employee requests. Agents must discover the relevant Oracle Fusion, Gmail, Drive, Sheets, and Slack evidence, reconcile records, calculate and compare options, then execute only the supported outcome.

Source benchmark Methodology

Manufacturing operations

Model performance

1 recorded configuration · 100 source tasks

Updated
Higher is better · 0–100 · execution errors score 0
C
Codexgpt-5.6-luna
76.2 FactoryScore1 / 100 strict pass · 100 graded · 0 execution errors
Full benchmark100 tasks attempted
View run
Run details
Evaluation
FactoryBench-100 · gpt-5.6-luna full run
Environment
Harbor 0.21.0 · Codex 0.151.0 · max reasoning
Grader
FactoryBench-100 deterministic verifier
Verifier
factorybench-100:3.3.5
Coverage
100 of 100 source tasks attempted; 100 graded.
Open the run record (per-task verdicts)

FactoryScore is the mean per-task score from the release's own verifier over the full roster. Strict passes are that verifier's all-or-nothing verdicts (version under Run details); the per-task verdicts are in the run record under Run details.

About FactoryBench-100

What it measures

Consult the upstream task verifier.

Scope & limitations

Listed in the catalog; not treated as independently established or famous. Catalog task counts do not prove validated coverage.

Evaluation environment

Dataset-specific harness qualification required.

Source

Original benchmark

Dataset & task catalog ↗