Skip to benchmark
Blobfish AI
BenchmarksTalk to usOpen Studio
Blobfish ResearchHubBench · rolling release4/13 families live

Every benchmark domain on Harbor Hub, rebuilt as an executable decision world.

288 open-source agent evaluations on hub.harborframework.com cluster into 16 professional domains. HubBench gives each cluster a Blobfish family: a stateful world behind provider-shaped MCP tools and a terminal CLI, where an ordinary employee question is only answerable through a dependent chain of evidence — and a deterministic verifier grades the whole chain, never just the final number.

Inspect the released families See domain coverage
Back to the portfolioReasoning-chain standard ↗
4released families
32qualified tasks
288hub datasets mapped
16domain clusters
0LLM grading calls
Measured controls first

No model score is shown until a full run on an exact release exists. Every released task passed the same admission gate before it counted: the reasoning-chain audit graded the full dependent chain, the reference solution scored a HubScore of exactly 100, a second execution reproduced that episode byte-for-byte, and all 10 negative-control attack types were rejected.

Across the 4 released families the qualification harness executed 448 isolated episodes: 32/32 oracle passes at a mean HubScore of 100, 320 negative-control episodes across 10 attack types with 0 false accepts, byte-identical determinism replays on every task, and 64/64 mutation omissions detected.

Inspect the admission gate →

Domain coverage

One family per domain cluster.

The cluster map is measured from the live Harbor Hub catalog (288 public datasets, observed 2026-08-29). A cluster counts as covered only when an executable, oracle-proven family exists for it — never on a name match.

  • HealthcareClinicOpsReleased · 8/8 qualified

    When can this patient realistically get the infusion, given the order, the drug on hand, the chair schedule, and the supplier's delivery window?

    Inspect the evidence
  • Terminal OperationsHostOpsReleased · 8/8 qualified

    Can we recover last night's failed deploy artifacts before the 09:00 release window, and what does it cost us if we rebuild instead?

    Inspect the evidence
  • Data Engineering AnalyticsDataDeskReleased · 8/8 qualified

    Why does Monday's revenue dashboard disagree with finance, and can we correct it before the board pack without re-running the full backfill?

    Inspect the evidence
  • Reasoning Knowledge QaResearchDeskReleased · 8/8 qualified

    What is the exact figure we can cite for last quarter's churn in the investor letter, and which alternative definitions would change it?

    Inspect the evidence
  • Computer Use GuideskopsPlanned

    Can we move the offsite to the week the venue has space, and what does that do to the travel budget and the attendees already booked?

  • Web Product DesignwebstudioPlanned

    Can the pricing page change ship this week without breaking the design system, and what does the fallback look like?

  • Policy Compliance Instruction FollowingpolicydeskPlanned

    Which of today's access requests can be approved under the current policy, which need an exception, and which must be refused outright?

  • It Operations ObservabilityitsmdeskPlanned

    Can we take the payments API down for the patch tonight, or does the error budget and the change freeze push it to the weekend?

  • SecuritysecopsPlanned

    Is this alert a real credential compromise, what has to be revoked right now, and what can wait for the owner's confirmation?

  • Customer Workplace AgentsworkplacePlanned

    Can the customer's escalation be closed this sprint with the people we have, and what do we tell them if not?

  • Scientific ResearchscilabPlanned

    When can the assay be re-run with a valid control, given the reagent lots on hand, the instrument calibration window, and the supplier's shipment?

  • Manufacturing Engineering DesigndesignopsPlanned

    Can the bracket revision be released to production this week without invalidating the certified assembly, and what does the interim look like?

  • Software EngineeringrepodeskPlanned

    Can the reported regression be fixed and shipped before the customer's cutover, and what is the safe fallback if the fix misses CI?

Released families

Four worlds, thirty-two qualified decisions.

Each family is a self-contained SQLite world behind provider-shaped tools — reachable as a stateful MCP server or a terminal CLI — with its own schema, seeded state, negative controls, and sealed deterministic verifier. Every task instruction below is the exact released text.

ClinicOpsHealthcare

v1.0.0 · fingerprint 505775309940

When can this patient realistically get the infusion, given the order, the drug on hand, the chair schedule, and the supplier's delivery window?

Infusion operations decisions against a FHIR-shaped EHR, chair scheduling, pharmacy inventory, supplier confirmations, and approval records.

  • order
  • dose requirement
  • on-hand minus expired
  • chair schedule
  • supplier lead time
  • alternatives
Qualified tasks
8/8
State tables
19
Provider tools
34
Evidence files / task
29–31
Deterministic criteria / task
61–66
Graded answer fields / task
24–27
Evidence reads before decision
26
Source systems consulted
9

Qualification: 112 isolated episodes — 8/8 oracle passes at mean HubScore 100, 80 negative-control episodes across 10 attack types with 0 false accepts, 8/8 byte-identical replays, and 16/16 mutation omissions detected.

Browse all 8 released task instructions
  • clinicops-001planinfusion operations coordinator

    Book Teodor Delacroix's infliximab induction

    Dr. Okafor's team wants Teodor Delacroix started on infliximab and the family keeps asking for a date. Before anyone promises them anything I need the real picture: what his order works out to at his current weight, what we genuinely have on the shelf once you set aside the quarantined and short-dated vials, whether Cascadia's confirmation changes the timing, and the first chair session that does not bump another patient. If the evidence supports it, book the chair and leave the care coordinator a draft she can use to call the family. Anything that needs the nursing director or the pharmacy director should be flagged, not assumed.

    29 evidence files65 criteria27 graded fields31 reference tool callschain depth 8
  • clinicops-002quantityinfusion pharmacy buyer

    Restock pembrolizumab for the week of 16 March

    Next week's oncology clinic has several pembrolizumab patients on the books and Priya wants the restock order placed today rather than Friday. Work out how many vials the scheduled doses actually need, what part of our stock can legitimately be used for those dates, and how many we should be ordering from Meridian under the approval she already signed. Place the order if it fits, and draft the note for the oncology charge nurse so she knows what is arriving and when. If something would need the director or a partner site, say so instead of doing it.

    29 evidence files61 criteria24 graded fields31 reference tool callschain depth 8
  • clinicops-003scheduleinfusion operations coordinator

    Fit Anneliese Varga's first rituximab infusion this week

    Anneliese Varga's first rituximab infusion is supposed to happen this week and Dr. Mbeki asked for Thursday. It is a long first-dose infusion with monitoring, so I need to know whether Thursday genuinely fits on a chair with first-dose nursing cover without displacing the oncology blocks, and if it does not, the earliest day that does. Confirm the drug is on hand for it too. Book the pending appointment where it truly fits, and leave a note for Dr. Mbeki's coordinator with the date, what constrained it, and what the alternatives would have cost or required.

    30 evidence files62 criteria24 graded fields31 reference tool callschain depth 8
  • clinicops-004planinfusion pharmacy buyer

    Get Kwame Boateng's IVIG in before neurology's deadline

    Neurology is pushing to get Kwame Boateng's IVIG in quickly because his weakness is progressing. Dr. Lindqvist's order is weight-based and pharmacy tells me part of our immune globulin is sitting in quarantine after the fridge alarm. I need to know how much the order really needs, how much of what we hold can actually be used, whether Meridian's standard or expedited delivery gets us there in time for the date neurology gave, and what borrowing from the partner hospital would involve. Place the order that is actually approved, then draft the update for neurology and scheduling.

    29 evidence files64 criteria27 graded fields31 reference tool callschain depth 8
  • clinicops-005quantityinfusion pharmacy buyer

    Transfer ocrelizumab from Eastgate for this week's doses

    We have two ocrelizumab patients at the infusion center this week and the main pharmacy is nearly out. Eastgate says they have stock, but some of it is spoken for and one lot looked old on the last count. Tell me exactly how many vials the two doses need, how many are already usable here, how many can legitimately come over from Eastgate, and whether buying direct instead is the better call. Schedule the transfer that the evidence supports and draft the message to the Eastgate pharmacist so the courier pickup is not a surprise.

    30 evidence files61 criteria24 graded fields31 reference tool callschain depth 8
  • clinicops-006scheduleinfusion operations coordinator

    Re-home the natalizumab series stranded by the CHAIR-1 outage

    The pump on chair one failed inspection and biomed has it out until the twentieth, which strands the natalizumab patients booked on it this week. Neurology is firm that none of them can slip past a week beyond their due date. Figure out how much chair time those infusions need, what is honestly open on the other two chairs without touching the oncology blocks, and how far into next week the last of them lands. Move the first affected appointment to the session you settle on, and leave the scheduling lead a note that lays out the rest and any option that would need Dr. Hale.

    31 evidence files63 criteria25 graded fields31 reference tool callschain depth 8
  • clinicops-007planinfusion operations coordinator

    Advance Rosalind Achterberg's abatacept dose before travel

    Rosalind Achterberg is leaving the country on the twenty-first for six weeks and Dr. Mbeki has agreed her abatacept can be given early. Her dose depends on her weight band, pharmacy says the vials on the shelf are a mix of good, short-dated, and one that was pulled after a temperature issue, and Meridian has quoted two delivery dates. I need to know whether we can dose her before she travels, on what day, and at what cost, and whether the short-dated stock could be used instead. Rebook her appointment accordingly and draft the note for the rheumatology nurse.

    30 evidence files66 criteria27 graded fields31 reference tool callschain depth 8
  • clinicops-008quantityinfusion pharmacy buyer

    Replace tocilizumab after the fridge excursion before Monday's clinic

    The pharmacy refrigerator drifted warm over the weekend and the tocilizumab for Monday's rheumatology clinic was inside. The manufacturer's stability letter is on the drive along with the excursion log, and the three Monday patients are already booked. Work out how many vials Monday needs, which of the exposed lots the letter still covers, how many replacement vials we have to order and by which delivery option so the clinic runs on time, and what it would take to use the uncovered lot instead. Place the replacement order that is within the approval and draft the note for the clinic lead.

    31 evidence files61 criteria24 graded fields31 reference tool callschain depth 8

HostOpsTerminal Operations

v1.0.0 · fingerprint 9c8e98d26d07

Can we recover last night's failed deploy artifacts before the 09:00 release window, and what does it cost us if we rebuild instead?

Host-operations recovery decisions against a Linux service inventory, cron/CI scheduler, backup catalog with retention and vendor retrievals, release build farm, and approval records.

  • job schedule
  • backup manifest
  • retention policy
  • rebuild capacity
  • release window
  • alternatives
Qualified tasks
8/8
State tables
22
Provider tools
39
Evidence files / task
30–33
Deterministic criteria / task
62–67
Graded answer fields / task
24–27
Evidence reads before decision
27
Source systems consulted
11

Qualification: 112 isolated episodes — 8/8 oracle passes at mean HubScore 100, 80 negative-control episodes across 10 attack types with 0 false accepts, 8/8 byte-identical replays, and 16/16 mutation omissions detected.

Browse all 8 released task instructions
  • hostops-001planplatform operations coordinator

    Stage the Atlas 26.1 release after the failed nightly publish

    Last night's Atlas publish died halfway through and the 26.1 train is supposed to leave Thursday morning. Before the release meeting I need the honest picture: how many archive segments the release and its rollback baseline actually need at the current bundle size, how many of what the catalog shows are genuinely restorable once the scrub failures and the purge queue are set aside, whether Coldspur's retrieval changes the date, and what a from-source rebuild would really involve. Book the verification window that works and leave Freya a draft she can take into the meeting. Anything that needs Annika or Bisi should be flagged, not assumed.

    30 evidence files66 criteria27 graded fields32 reference tool callschain depth 8
  • hostops-002quantityplatform operations coordinator

    Size the payments rollback retrieval for the week of 20 April

    Next week's payments release trains are on the farm calendar and Imogen wants the rollback-set retrieval placed today rather than Thursday. Work out how many archive segments the scheduled verifications actually need, which part of the nearline holdings can legitimately be staged for those dates, and how many we should be pulling from Ironhold under the approval she already signed. Place the restore job if it fits, and draft the note for Deepak so his team knows what arrives and when. If something would need Bisi or a DR copy, say so instead of doing it.

    30 evidence files62 criteria24 graded fields32 reference tool callschain depth 8
  • hostops-003scheduleplatform operations coordinator

    Fit the ledger restore rehearsal before the audit deadline

    The quarterly ledger restore rehearsal is supposed to happen this week and Matteo asked for Thursday. It is a long continuous run with evidence capture, so I need to know whether Thursday genuinely fits on a runner with the isolated VLAN without displacing the compliance batch, and if it does not, the earliest day that does. Confirm the snapshot segments are actually restorable too. Book the pending reservation where it truly fits, and leave a note for Matteo with the date, what constrained it, and what the alternatives would have cost or required.

    31 evidence files63 criteria24 graded fields32 reference tool callschain depth 8
  • hostops-004planplatform operations coordinator

    Restore the audit-log week before the regulator session

    Compliance is pushing to get a week of audit logs restored and indexed quickly because the regulator session is Monday. The volume is metered per day and part of what we hold was flagged by the scrub after the power event. I need to know how many day segments the week really needs at the current metering, how much of our catalog can actually be used, whether Ironhold's standard or expedited retrieval gets us there in time for the date compliance gave, and what pulling the DR replica would involve. Place the restore job that is actually approved, then draft the update for Matteo and the on-call channel.

    30 evidence files65 criteria27 graded fields32 reference tool callschain depth 8
  • hostops-005quantityplatform operations coordinator

    Copy telemetry segments from Drayton for this week's replays

    Two telemetry replay validations run on the farm this week and the nearline store is nearly empty. Drayton says they have segments, but some are spoken for and one set looked lapsed on the last sweep. Tell me exactly how many segments the two replays need, how many are already usable here, how many can legitimately come over from the DR store, and whether retrieving direct from Coldspur is the better call. Schedule the copy the evidence supports and draft the message to the Drayton operator so the replication pickup is not a surprise.

    31 evidence files62 criteria24 graded fields32 reference tool callschain depth 8
  • hostops-006scheduleplatform operations coordinator

    Re-home the patch verifications stranded by the RUNNER-1 outage

    Runner one's hypervisor failed this morning's attestation and it is fenced until the twenty-fourth, which strands the kernel patch verifications booked on it this week. Security is firm that none of them can slip past a week beyond their due date. Figure out how much runner time those runs need, what is honestly open on the other two runners without touching the protected blocks, and how far into next week the last of them lands. Move the first affected reservation to the window you settle on, and leave Amara a note that lays out the rest and any option that would need Marta.

    32 evidence files64 criteria25 graded fields32 reference tool callschain depth 8
  • hostops-007planplatform operations coordinator

    Advance the catalog rehearsal before the power remediation

    The primary data centre goes dark for power remediation from the twenty-fifth and the catalog migration rehearsal is currently booked after that. The change owner says it can come forward. Its baseline depends on the dataset tier, the nearline segments are a mix of good, purge-queued, and one the scrub flagged, and Ironhold has quoted two retrieval dates. I need to know whether we can stage the rehearsal before the cutoff, on what day, and at what cost, and whether the purge-queued set could carry it instead. Rebook the reservation accordingly and draft the note for June.

    32 evidence files67 criteria27 graded fields32 reference tool callschain depth 8
  • hostops-008quantityplatform operations coordinator

    Replace the scrub-flagged storefront segments before Monday's trains

    The morning scrub flagged part of the storefront rollback archive after last week's power event, and Monday has three ten-gig-class train verifications on the calendar plus the edge-widget train on its own class. Tell me how many segments Monday genuinely needs with storefront-web on its metered size, which sets can still be staged under the new durability letter rather than the old one, and how many segments to pull from Coldspur under Imogen's approval. Place that retrieval, then draft the note for Freya so the storefront owners know what is arriving and what stays quarantined.

    33 evidence files62 criteria24 graded fields32 reference tool callschain depth 8

DataDeskData Engineering Analytics

v1.0.0 · fingerprint 6708188e6bc5

Why does Monday's revenue dashboard disagree with finance, and can we correct it before the board pack without re-running the full backfill?

Data engineering decisions against a dbt-style warehouse catalog, pipeline run history, vendor feed deliveries, freshness SLAs, batch-window capacity, and finance reconciliation controls.

  • model lineage
  • run history
  • source counts
  • freshness SLA
  • backfill capacity
  • alternatives
Qualified tasks
8/8
State tables
19
Provider tools
33
Evidence files / task
30–32
Deterministic criteria / task
60–62
Graded answer fields / task
23–24
Evidence reads before decision
26
Source systems consulted
8–9

Qualification: 112 isolated episodes — 8/8 oracle passes at mean HubScore 100, 80 negative-control episodes across 10 attack types with 0 false accepts, 8/8 byte-identical replays, and 16/16 mutation omissions detected.

Browse all 8 released task instructions
  • datadesk-001quantityanalytics engineer oncall

    Reconcile Monday's revenue dashboard to the W10 close

    Finance flagged Monday's revenue dashboard within an hour of the board pack draft going out: it disagrees with Corin's week-ten close workbook and nobody can say which number is right. Work out what the certified table genuinely supports once the invalid, duplicate, and late Skyfield rows are set aside, how large the true shortfall against the published control total is, and whether waiting for the corrected file, expediting it, or posting the signed reconciling entry gets us right before Wednesday's pack. Post only what the evidence supports, and leave Corin a draft note that explains the number. Anything needing Ingrid should be flagged, not assumed.

    31 evidence files60 criteria23 graded fields31 reference tool callschain depth 8
  • datadesk-002plananalytics engineer oncall

    Backfill the margin partitions poisoned by Harrier's reference cut

    Harrier shipped us a bad landed-cost reference cut and the margin mart has been quietly wrong for days; the CFO reviews margins Friday morning. Establish which daily partitions in the trailing window are still trustworthy, which ones consumed the contaminated file, and how many genuinely need reloading rather than rebuilding everything. Then test whether the vendor's corrected files, standard or expedited, actually change when a backfill can run given the batch windows we're allowed to touch. Schedule the correction that is actually approved and draft the note for the pricing team, flagging anything that would need Femi's sign-off.

    30 evidence files62 criteria24 graded fields31 reference tool callschain depth 8
  • datadesk-003scheduleanalytics engineer oncall

    Window the Q1 revenue full refresh before the audit extract

    The quarterly full refresh of the revenue mart is still sitting unscheduled and the audit evidence extract runs Thursday evening. Femi asked for Tuesday. Before promising that, check how long the refresh really takes end to end, whether Tuesday honestly offers that much contiguous room on a cluster we may use without touching the close loads, and if not, the earliest day that does. Window the pending job where it truly fits, and leave the analytics manager a note with the date, what constrained it, and what the faster routes would have cost or required.

    31 evidence files62 criteria24 graded fields31 reference tool callschain depth 8
  • datadesk-004quantityanalytics engineer oncall

    Reverse the double-posted Bluecrest settlement batch

    Bluecrest re-sent Wednesday's settlement batch under a fresh filename on Thursday night, our dedup stage happened to be down, and Monday's published settlements snapshot now runs hot against the processor statement. Establish exactly how many rows are genuine duplicates as opposed to ordinary rejects, what the certified table should say for the week, and the cleanest authorized way to correct the published number before finance signs off Wednesday. Post the reversal that matches the evidence and the signed approval, then draft the explanation for Corin. Escalate rather than improvise anything touching expedite fees.

    32 evidence files60 criteria23 graded fields31 reference tool callschain depth 8
  • datadesk-005plananalytics engineer oncall

    Reload the attribution days broken by Cormorant's schema cutover

    Cormorant flipped their clickstream to a new event schema without notice and our parser has been silently dropping the consent fields ever since, so attribution is wrong going into Thursday's growth review. Work out which event days in the trailing window still stand, which were parsed from the new schema, and how many days genuinely need reprocessing. Then see whether their standard or expedited re-export actually lands in time once you account for the warehouse windows we can use. Schedule the reload the approval covers, and draft the note for the growth lead, flagging any full-history option for Ingrid.

    30 evidence files62 criteria24 graded fields31 reference tool callschain depth 8
  • datadesk-006scheduleanalytics engineer oncall

    Re-window the certified loads stranded by the WH-XL outage

    Platform ops pulled cluster XL this morning for a storage firmware patch that runs through next Thursday, stranding the four certified finance loads that live in its night window. None of them may run more than five business days past its cycle date or the mart is decertified. Figure out how much window time the loads need, what is honestly open on the standard cluster without touching replication, and how far into next week the last one lands. Re-window the first stranded load where it fits and leave warehouse ops a note covering the rest and anything needing Femi.

    32 evidence files62 criteria24 graded fields31 reference tool callschain depth 8
  • datadesk-007quantityanalytics engineer oncall

    Close the inventory snapshot gap before the S&OP review

    Saturday's inventory snapshot came in two regional files short and the S&OP review uses it Tuesday afternoon. The northwest re-send is already validated in staging and rides tonight's load, but the southeast extract is still with Harrier. Establish what the snapshot actually holds against finance's SKU-location control, how much tonight's load genuinely recovers, and the residual that only an adjustment or a vendor re-cut can close in time. Post the entry the approval supports for exactly that residual, and draft the update for Rei, flagging the expedite route since its fee is nobody's to wave through here.

    31 evidence files60 criteria23 graded fields31 reference tool callschain depth 8
  • datadesk-008plananalytics engineer oncall

    Rebuild the February revenue partitions before the audit hand-off

    Skyfield's storefront rollout dropped cancellation records from our order files for most of late February, and the restated revenue for that window has to be certified before the auditors take their extract Monday morning. Determine which February partitions still stand, which ones need rebuilding from corrected exports, and whether paying to expedite the vendor's files buys any real time once the month-end close windows are respected. Reserve the rebuild the approval actually covers, and draft the hand-off note for Corin and the audit liaison. A shortcut adjustment to a closed period is Femi's call, not ours.

    31 evidence files62 criteria24 graded fields31 reference tool callschain depth 8

ResearchDeskReasoning Knowledge Qa

v1.0.0 · fingerprint c13aa87a4a93

What is the exact figure we can cite for last quarter's churn in the investor letter, and which alternative definitions would change it?

Multi-source internal research across versioned knowledge, metric definitions, source provenance, ranked search, approvals, review capacity, published claims, and evidence packets.

  • definition policy
  • source tables
  • exclusions
  • prior letter
  • sign-off authority
  • alternatives
Qualified tasks
8/8
State tables
17
Provider tools
32
Evidence files / task
30
Deterministic criteria / task
60–62
Graded answer fields / task
27–28
Evidence reads before decision
26
Source systems consulted
10

Qualification: 112 isolated episodes — 8/8 oracle passes at mean HubScore 100, 80 negative-control episodes across 10 attack types with 0 false accepts, 8/8 byte-identical replays, and 16/16 mutation omissions detected.

Browse all 8 released task instructions
  • researchdesk-001quantityresearch analyst

    Resolve the exact Q2 customer-logo churn figure

    The operating review draft quotes two different Q2 customer-logo churn figures, and the board secretary needs one defensible number before Thursday. Resolve the active article and metric definition, separate genuine cancellations from migrations and administrative closures, and reconcile the ledger, cohort export, and methodology register to the exact supported basis-point figure. Compare publishing that figure now with waiting for finance reconciliation or reusing the retired definition. Publish only the claim covered by Mina's approval and leave the board secretary a draft explanation of why the older headline differs.

    30 evidence files60 criteria27 graded fields31 reference tool callschain depth 7
  • researchdesk-002planresearch analyst

    Assemble the renewal-risk evidence packet

    Sales strategy is challenging the renewal-risk paragraph in next week's planning memo because the current knowledge article links to a mix of live evidence, an expired forecast, and a search snippet with no source context. Establish the current definition and the minimum source coverage the memo requires, remove superseded or conflicting material, and quantify the remaining evidence gap. Compare a verified packet with waiting for the archive refresh or publishing snippets alone. Create the approved packet with the exact included and excluded sources, then draft a note explaining what the packet supports and what still needs separate authority.

    30 evidence files61 criteria28 graded fields31 reference tool callschain depth 8
  • researchdesk-003scheduleresearch analyst

    Reserve the net-retention methodology review

    Finance and customer success disagree over the net-retention definition attached to the investor FAQ, and the answer cannot move forward without a methodology review. Determine which article revision and source set are current, distinguish ordinary reviewer availability from protected close work, and size the meeting from the documented review requirement rather than a calendar title. Compare the qualified slot with a generalist opening and a faster protected session. Reserve the authorized specialist window without displacing protected work, then draft the FAQ owner a note with the chosen timing, decisive definition issue, and any escalation still required.

    30 evidence files62 criteria28 graded fields31 reference tool callschain depth 7
  • researchdesk-004quantityresearch analyst

    Correct the support-deflection claim

    The product launch brief says self-service deflected nearly eight out of ten support requests, but operations says the headline includes bot handoffs and duplicate contacts that the current definition excludes. Locate the authoritative article and current measurement period, reconcile the event ledger, help-center cohort, and exclusion register, and compute the exact supported basis-point figure. Compare publishing the corrected result, waiting for another event replay, and keeping the retired headline. Persist only the approved current-definition claim and leave product marketing a draft that makes the definition change and residual gap explicit.

    30 evidence files60 criteria27 graded fields31 reference tool callschain depth 7
  • researchdesk-005planresearch analyst

    Build the market-expansion eligibility packet

    Corporate development wants the Southeast Asia expansion brief to claim ten markets are launch-ready, while the current research room contains partner attestations, duplicated regulatory summaries, and two expired tax opinions. Establish the active readiness definition and the source positions the brief actually requires, reconcile which evidence remains independent and current, and quantify the uncovered positions. Compare an approved verified packet with waiting for refreshed opinions or relying on ranked snippets. Create the packet using only supported sources and draft the strategy lead a note that names the remaining evidence gap and the authority boundary.

    30 evidence files61 criteria28 graded fields31 reference tool callschain depth 8
  • researchdesk-006quantityresearch analyst

    Resolve voluntary attrition under the current definition

    People leadership found three attrition percentages in the quarterly talent narrative, each using a different treatment of internal transfers, retirements, and acquired employees. Resolve the active knowledge revision, quarter boundary, and approved population definition; reconcile the HR event export, opening-headcount control, and exclusions register to the exact supported basis-point figure. Compare publishing the current result with waiting for payroll close or reusing the legacy calculation. Publish only the claim within the signed scope and draft People leadership an explanation that distinguishes the current metric from the retired headline.

    30 evidence files60 criteria27 graded fields31 reference tool callschain depth 7
  • researchdesk-007scheduleresearch analyst

    Reserve the emissions-intensity assurance review

    The sustainability report team needs assurance on an emissions-intensity claim after discovering that the current article and last year's workbook use different facility and renewable-credit boundaries. Resolve the operative definition and linked source set, determine the documented specialist time required, and distinguish usable review capacity from protected regulatory work. Compare the qualified assurance slot with a generalist opening and a faster protected session. Reserve only the approved specialist window, preserving every protected slot, and draft the report owner a note with the review date, methodology boundary, and any faster path that would require separate approval.

    30 evidence files62 criteria28 graded fields31 reference tool callschain depth 7
  • researchdesk-008planresearch analyst

    Create the onboarding-time root-cause packet

    Customer operations wants to tell the executive team why enterprise onboarding time rose, but the knowledge search mixes current workflow evidence with a retired implementation survey and duplicate regional extracts. Establish the active time-to-value definition and the evidence positions needed for a causal statement, remove stale or non-independent sources, and quantify what remains unsupported. Compare a verified root-cause packet with waiting for the survey refresh or publishing search snippets as if they were evidence. Create the authorized packet from the exact current sources and draft operations a note that separates supported drivers from unresolved hypotheses.

    30 evidence files61 criteria28 graded fields31 reference tool callschain depth 8

Sample decision

The verifier grades the chain, the options, and the exact answer.

From ClinicOps task clinicops-001: the agent must reach the decision through 26 required evidence reads, then commit only the approved option. All three alternatives carry an exact outcome date, incremental cost, and authority status — and every one of those values is graded.

Dr. Okafor's team wants Teodor Delacroix started on infliximab and the family keeps asking for a date. Before anyone promises them anything I need the real picture: what his order works out to at his current weight, what we genuinely have on the shelf once you set aside the quarantined and short-dated vials, whether Cascadia's confirmation changes the timing, and the first chair session that does not bump another patient. If the evidence supports it, book the chair and leave the care coordinator a draft she can use to call the family. Anything that needs the nursing director or the pharmacy director should be flagged, not assumed.

Binding constraint: drug readiness versus the non-displacing chair calendar; quarantined and short-dated INFLIX-100 vials = 5 VIAL; uncovered scope = 3 VIAL. The oracle resolves this in 31 tool calls; the reference is one valid solution, not a required call order. standard_delivery_plan is the best currently authorized response that satisfies drug readiness versus the non-displacing chair calendar and lands 1 day(s) on or before the control date.

  • standard_delivery_planRecommended

    standard delivery plan uses only dispensable reference-product vials plus the standard Cascadia shipment, lands on the first non-displacing chair session on 2026-03-19, one day before the control date, at no incremental cost.

    outcome 2026-03-19USD 0SUPPORTED AND APPROVED
  • expedite_supplier_shipmentAVAILABLE NOT RECOMMENDED

    expedite supplier shipment would have vials on the shelf by 2026-03-16 and adds USD 210, but the first non-displacing chair session is still 2026-03-19, so it saves 0 days.

    outcome 2026-03-19USD 210FEASIBLE WITH INFERIOR TRADEOFF
  • add_extended_evening_sessionADDITIONAL APPROVAL REQUIRED

    add extended evening session would finish 1 day earlier than standard delivery plan and adds USD 480 of nursing overtime, but extended sessions need the nursing director's separate approval, which AP-CO-0101 does not carry.

    outcome 2026-03-18USD 480SEPARATE APPROVAL OR POLICY EXCEPTION REQUIRED

Methodology

Admission is an audit, not a claim.

HubBench families are admitted under the same published reasoning-chain standard as the rest of the Blobfish portfolio, where all 524 authored and measured tasks currently pass (524/524 across 11 benchmarks).

  1. 01 · Chain audit

    The full chain is graded

    Thirteen hop classes cover schedule, quantity, specification, constraint, derivation, option, and answer hops. A task passes only when the audit finds the dependent chain — graded intermediates, three costed and authority-checked alternatives, and an exact multi-field answer — with zero LLM-judge calls.

  2. 02 · Oracle proof

    Solvable, exactly, twice

    The reference agent must score exactly 100 on every task, and a second full execution must reproduce the episode byte-for-byte. Reference replay proves solvability and determinism — it is never presented as model difficulty.

  3. 03 · Negative controls

    10 attack types, zero false accepts

    No-ops, shortcuts, state-only writes, incomplete reads, write-before-read, missing readbacks, unauthorized writes, wrong values, wrong decisions, and wrong evidence all run against every task and must all be rejected before the family is released.

  4. 04 · Real state

    Writes persist, readbacks verify

    Every task ends in an authorized state change on the world's SQLite tables plus a persisted handoff, verified by post-write readbacks. The sealed verifier is unreachable from the agent's tool surface.

Task records, qualification counts, and chain-audit measurements on this page are the release evidence. Harbor, Hugging Face, and public repository distribution are queued and will be linked here when live. Observed 2026-08-29; 593,031 upstream tasks across 288 hub datasets informed the cluster map. Seed shapes are named per family for provenance; every world, schema, and task is clean-room Blobfish authorship.

Read the executable standard ↗See the standard applied to FactoryBench-100 →

Next

Nine more families, one per remaining cluster.

The engine, controls, and admission gate are family-agnostic. Each remaining Harbor Hub domain cluster receives the same treatment: an executable world, at least 8 chain-qualified tasks, and published evidence — toward 104 tasks across 13 families.

Explore the full portfolio Browse Harbor Hub ↗