Skip to datasets
Blobfish AI
BenchmarksTalk to usOpen app

Blobfish datasets

Datasets built
by Blobfish.

Train and evaluate agents on our released tasks and stateful environments. Inspect the world, task, and verification evidence behind each dataset.

Watch tasks, results & verification ↓Watch real footage + robot gyms ↓Watch human POV ↓Explore agent datasets ↓View model results →
13Blobfish datasets
1,171Released tasks
1,171/1,171Tasks with a passing oracle replay

TinyFish · Native business workflows

The task. The result. The proof.

Select a task to read the employee request, watch the continuous recording, and inspect how its saved outcome was verified.

40 scripted reference walkthroughs in fictional business fixtures. Autonomous model performance and minimum necessary service hops remain unmeasured.

40 tasks

I01 · Incident commander · Kubernetes / Headlamp / EspoCRM / GLPI / Gitea / OpenObserve / Grafana / BookStack

Investigate checkout failures and customer impact

Investigate INC-1042 checkout failures, identify the affected customers and platforms, and update the incident with evidence.

Evidence to reconcile

  • Executed request logs and region/platform denominators
  • Deployed Kubernetes revision and the introducing pull request
  • Affected legal customers and current incident policy

Native saved outcome verified · 82 checks · 200 seconds

INC-1042 is assigned to Checkout Operations with the failing requests, five region/platform denominators, affected legal customers, deployed revision and causal pull request recorded.

How we verified it

  1. Check the saved incident's ownership, responsible group and single follow-up.
  2. Match each denominator and legal customer against native evidence; distinguish validation rejection from the response error.
  3. Check the deployed source revision and introducing pull request recorded in the follow-up.
  4. Compare before/after native inventories to preserve sibling incidents, approvals, CRM, accounting, deployments, source and logs.
Native verification reportOpen original MP4Original qualification report
Inspect all 82 native checks
Native checkResultSaved value
target incident persistsPassedSee verification report
preserve incident idPassedSee verification report
preserve incident entities_idPassedSee verification report
preserve incident namePassedSee verification report
preserve incident datePassedSee verification report
preserve incident closedatePassedSee verification report
preserve incident solvedatePassedSee verification report
preserve incident takeintoaccountdatePassedSee verification report
preserve incident users_id_recipientPassedSee verification report
preserve incident requesttypes_idPassedSee verification report
preserve incident contentPassedSee verification report
preserve incident itilcategories_idPassedSee verification report
preserve incident typePassedSee verification report
preserve incident slas_id_ttrPassedSee verification report
preserve incident slas_id_ttoPassedSee verification report
preserve incident slalevels_id_ttrPassedSee verification report
preserve incident time_to_resolvePassedSee verification report
preserve incident time_to_ownPassedSee verification report
preserve incident begin_waiting_datePassedSee verification report
preserve incident sla_waiting_durationPassedSee verification report
preserve incident ola_waiting_durationPassedSee verification report
preserve incident olas_id_ttoPassedSee verification report
preserve incident olas_id_ttrPassedSee verification report
preserve incident olalevels_id_ttrPassedSee verification report
preserve incident ola_tto_begin_datePassedSee verification report
preserve incident ola_ttr_begin_datePassedSee verification report
preserve incident internal_time_to_resolvePassedSee verification report
preserve incident internal_time_to_ownPassedSee verification report
preserve incident waiting_durationPassedSee verification report
preserve incident close_delay_statPassedSee verification report
preserve incident solve_delay_statPassedSee verification report
preserve incident actiontimePassedSee verification report
preserve incident is_deletedPassedSee verification report
preserve incident locations_idPassedSee verification report
preserve incident date_creationPassedSee verification report
preserve incident tickettemplates_idPassedSee verification report
preserve incident externalidPassedSee verification report
assigned or pending dispositionPassedSee verification report
actor owns saved updatePassedSee verification report
native responsible groupPassedSee verification report
one actual incident followupPassedSee verification report
followup attaches to targetPassedSee verification report
followup is authored by ordinary actorPassedSee verification report
nonempty saved evidencePassedSee verification report
matching denominator ap-southeast/mobile_webPassedSee verification report
matching denominator eu-west/iosPassedSee verification report
matching denominator eu-west/webPassedSee verification report
matching denominator us-east/androidPassedSee verification report
matching denominator us-east/iosPassedSee verification report
correct affected legal customer Cedar Mobility USPassedSee verification report
correct affected legal customer Nimbus Pacific Pte LtdPassedSee verification report
separate validation rejectionPassedSee verification report
identify response errorPassedSee verification report
identify actual deployed sourcePassedSee verification report
identify causal PRPassedSee verification report
preserve Ticket/2PassedSee verification report
preserve Ticket/3PassedSee verification report
preserve Ticket/4PassedSee verification report
preserve Ticket/5PassedSee verification report
preserve Ticket_User/1PassedSee verification report
preserve Ticket_User/2PassedSee verification report
preserve Ticket_User/3PassedSee verification report
preserve Ticket_User/4PassedSee verification report
preserve Ticket_User/5PassedSee verification report
preserve Ticket_User/6PassedSee verification report
preserve Ticket_User/7PassedSee verification report
preserve Ticket_User/8PassedSee verification report
preserve Ticket_User/9PassedSee verification report
preserve Ticket_User/10PassedSee verification report
no extra TicketPassedSee verification report
no unsolicited approvalsPassedSee verification report
groups affect only targetPassedSee verification report
no extra Ticket_UserPassedSee verification report
preserve native crmPassedSee verification report
preserve native erpPassedSee verification report
preserve native deploymentsPassedSee verification report
preserve native sourcePassedSee verification report
preserve native logsPassedSee verification report
preserve native template configurationPassedSee verification report
browser execution succeedsPassedSee verification report
reference visits its declared native surfacesPassedSee verification report
continuous 1080p reference footagePassedSee verification report
Watch the final saved records for all four outage tasks

A separate read-only recording shows the four persisted incident records. Before/after inventory checks confirm that viewing them changed no native records.

Unchanged-state verification
4 earlier failed attempts retained

Incomplete reference · I01-20261004T203027Z

  • actor owns saved update
  • one actual incident followup
  • followup attaches to target
  • followup is authored by ordinary actor
  • nonempty saved evidence
  • matching denominator ap-southeast/mobile_web
  • matching denominator eu-west/ios
  • matching denominator eu-west/web
  • matching denominator us-east/android
  • matching denominator us-east/ios
  • correct affected legal customer Cedar Mobility US
  • correct affected legal customer Nimbus Pacific Pte Ltd
  • separate validation rejection
  • identify response error
  • identify actual deployed source
  • identify causal PR
  • browser execution succeeds
  • reference visits its declared native surfaces
Failure verification · Retained attempt MP4

Incomplete reference · I01-20261004T204451Z

  • actor owns saved update
  • one actual incident followup
  • followup attaches to target
  • followup is authored by ordinary actor
  • nonempty saved evidence
  • matching denominator ap-southeast/mobile_web
  • matching denominator eu-west/ios
  • matching denominator eu-west/web
  • matching denominator us-east/android
  • matching denominator us-east/ios
  • correct affected legal customer Cedar Mobility US
  • correct affected legal customer Nimbus Pacific Pte Ltd
  • separate validation rejection
  • identify response error
  • identify actual deployed source
  • identify causal PR
  • browser execution succeeds
  • reference visits its declared native surfaces
Failure verification · Retained attempt MP4

Incomplete reference · I01-20261004T223624Z

  • one actual incident followup
  • followup attaches to target
  • followup is authored by ordinary actor
  • nonempty saved evidence
  • matching denominator ap-southeast/mobile_web
  • matching denominator eu-west/ios
  • matching denominator eu-west/web
  • matching denominator us-east/android
  • matching denominator us-east/ios
  • correct affected legal customer Cedar Mobility US
  • correct affected legal customer Nimbus Pacific Pte Ltd
  • separate validation rejection
  • identify response error
  • identify actual deployed source
  • identify causal PR
  • browser execution succeeds
  • reference visits its declared native surfaces
  • reference footage exists
Failure verification

Incomplete reference · I01-20261004T224422Z

  • native responsible group
  • preserve Group_Ticket/1
Failure verification · Retained attempt MP4
Recording identity and collection manifest

Original video SHA256

9ca8061f6a5a941d4588a45929f9457e133a750926c2e7d40dfaba2f4720708a

Public collection manifest · Published file hashes

ROBOFISH / ROBOT SIMULATIONS

Real footage. Robot gyms.

See the real human recording beside a robot gym preview. Choose a recording, an authored gym task, and Panda or ALOHA.

These recordings use authored scenes. Source-faithful reconstructions are still under review.

200robot videos
2robot types
6task families
Robot

Real footage

Original human POV excerpt · recording 173

Original recording ↗

Panda robot gym

Wipe region · Goal reached · scripted controller · authored scene

Gym recipe ↗

The gym tasks are authored reference scenarios. Their actions and timing have not been verified against the real recording.

More robot recordings

100 of 100 Panda recordings

Recorded simulations with scripted controllers. These videos do not demonstrate learned-policy performance or verified reconstruction of the human footage. Watch the original human POV recordings ↓

Asset attribution · Recording manifest

ROBOFISH / HUMAN POV

POV recordings.

Watch the human footage used to develop Robofish gyms. Each recording preserves the original viewpoint, hands, objects, and motion.

25POV recordings

Source footage: Eidon AI / tracker-pov, CC BY 4.0. These players show excerpts of the source recordings.

Get an agent dataset in one command.

Each release is a Harbor package that runs the world, the tasks and the verifier under Docker, and — where a mirror is published — a Hugging Face release pinned to the exact commit the evidence pages cite. World sources live on GitHub where the repository is public.

harbor run -d blobfishai/webbench
Every entry comes from the release record Blobfish published for that dataset; the counts on this page are the qualification results, not model scores.All Blobfish packages on Harbor ↗

Available now

Our released datasets.

13 datasets

Browser-only SaaS operations · Browser-only suite

WebBench v0.3.0

Browser-only correlated SaaS work over deterministic populated worlds: 137 tasks across 3 families, every one graded executably against the backend of the applications the agent touched — WebScore, no LLM judge.

Tasks
137 tasks
World
83 tables
Qualification
137/137 oracle passes · 0 false accepts across 822 negative controls
License
CC BY 4.0 task data; engine under Apache-2.0
Run
harbor run -d blobfishai/webbench
Explore dataset →Harbor release ↗Harbor-only release · no Hugging Face mirror yet

13 professional-domain families · Hub of domain families

HubBench v1.4.0

Thirteen oracle-proven professional-domain families inspired by the Harbor Hub inventory, with stateful worlds reachable as MCP servers, a REST API, a web console, and a terminal CLI. Source-specific adaptations are tracked separately.

Tasks
104 tasks
World
554 tools · 313 tables · 3,450 documents
Qualification
104/104 oracle passes · 0 false accepts
License
CC BY 4.0 dataset; code under Apache-2.0
Run
harbor run -d blobfishai/hubbench
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

Litigation · Domain suite

Blobfish Litigation Bench v1.0

100 long-horizon litigation tasks, 104–113 verified tool-calling steps each, graded by deterministic state diff — no LLM judge on the reward path.

Tasks
100 tasks
World
47 tools · 660 documents
Qualification
100/100 oracle passes
License
CC BY 4.0 dataset
Run
harbor run -d blobfishai/litigation
Explore dataset ↗Harbor release ↗Hugging Face ↗

Investment banking · Domain suite

DealBench-100 v1.0.0

100 deterministic long-horizon investment-banking tasks across ten synthetic deal worlds.

Tasks
100 tasks
World
36 tools · 19 tables · 440 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 500 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0
Run
harbor run -d blobfishai/dealbench-100-suite@v1.0.0

The Harbor registry's latest tag is v1.2.0; this card and the evidence page describe the v1.0.0 release.

Explore dataset →Harbor release ↗Hugging Face ↗

Software operations · Domain suite

DevOpsBench-100 v3.2.7

100 deterministic long-horizon DevOps/SRE tasks over one executable NovaCart world.

Tasks
100 tasks
World
97 tools · 72 tables · 30 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 1,400 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0
Run
harbor run -d blobfishai/devopsbench-100
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

Enterprise resource planning · Domain suite

ERPBench-100 v1.0.0

100 deterministic long-horizon ERP workflows across ten synthetic Oracle-Fusion-shaped tenants.

Tasks
100 tasks
World
52 tools · 31 tables · 460 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 600 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0
Run
harbor run -d blobfishai/erpbench-100-suite
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

Manufacturing operations · Domain suite

FactoryBench-100 v3.3.5

100 employee-grade manufacturing decisions with hidden investigation graphs and executable ERP outcomes.

Tasks
100 tasks
World
94 tools · 7 tables · 2,800 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 1,200 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0
Run
harbor run -d blobfishai/factorybench-100
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

Financial operations · Domain suite

LedgerBench-100 v3.4.2

100 deterministic corporate-finance tasks over a shared ERP world with real SEC XBRL filings.

Tasks
100 tasks
World
66 tools · 61 tables · 59 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 1,400 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0; SEC XBRL facts are US-government public-domain data
Run
harbor run -d blobfishai/ledgerbench-100
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

Semiconductor operations · Domain suite

SemiKongBench-100 v1.0.0

100 deterministic semiconductor exception workflows across ten synthetic fabs.

Tasks
100 tasks
World
38 tools · 24 tables · 380 documents
Qualification
100/100 oracle passes · 100/100 exact replays · 0 false accepts across 600 negative controls
License
CC BY 4.0 dataset; code under Apache-2.0; SemiKong design anchors under their published upstream terms
Run
harbor run -d blobfishai/semikongbench-100
Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

6 companion worlds · Companion-world portfolio

Blobfish DomainBench-24 v3.3.2

24 stateful tasks across 6 Blobfish-authored companion worlds, each a SQLite-backed MCP world with typed read/write tools.

Tasks
24 tasks
World
258 tools · 36 tables
Qualification
24/24 oracle passes
License
Terms included with the release (dataset card: license other); upstream benchmark names and source projects retain their own licenses
Run
harbor run -d blobfishai/domainbench-24
Explore dataset ↗Harbor release ↗Hugging Face ↗World sources on Hugging Face ↗

CRM operations · Source-inspired suite

Arc CRM 6 v0.1.2

Six source-locked CRM workflows. Four real interfaces. One isolated, stateful mock world per episode.

Tasks
6 tasks
World
33 tools · 33 documents
Qualification
6/6 oracle passes
License
Apache-2.0 independently authored code and synthetic fixtures
Run
harbor run -d blobfishai/arc-crm-6@sha256:fc9ca3b80d6a1a3c3e5bd1b975c54f1f3ada21b49668cb5f25e2912d67b69dfa -a oracle -e docker -n 1 -k 1 -r 0

Independently authored; inspired by a pinned upstream source with 0 source rows reproduced.

Explore dataset →Harbor release ↗Hugging Face ↗World source on GitHub ↗

In preview

On its way to a release.

1 dataset

Site reliability · Incident root cause

SRE World preview

Incident root-cause tasks cut from real recordings of a broken online shop. The agent gets a complaint and the shop's records and must say what broke and when.

Tasks
206 tasks
World
2 recorded shops · 9 verified incidents · 13 levers
Qualification
Reference answers score 1.0 in every packaged task run · 30 tasks flagged by our own checks, labelled not hidden
Model runs
56 runs by 5 models, every step published
Release
Not yet a Harbor release; answer keys are public so the work can be checked
Explore dataset →Every task and run →About SRE World →