All benchmarksScientific discovery

DiscoveryBench-102Evidence 2026-08-29Independent companionDataset page live

Can an agent turn scientific data into a reproducible result?

Scientific-programming tasks paired with an assay, compound, target, QC-result, and literature world.

Explore 4 companion tasks ↓World repository ↗Run on Harbor ↗

102upstream Harbor tasks

4independent companion tasks

28files per task room

59+criteria per task

Baseline leaderboard

Solvable, discriminating, and honest about what ran.

All companion tasks are solvable by their reference traces, exact replays, and reject thirteen task-applicable controls including correct state with wrong reasoning, wrong target, and keyword stuffing. No paid or frontier model run is claimed. These 4 tasks are Blobfish-authored companions, not conversions of the 102-task upstream suite. The first two rows are harness controls; the model leaderboard remains open until a version-pinned submission exists.

EntryRoleTasksPass rateMean rewardTool calls

Reference solverDeterministic trace replay; not a model rank.

Solvability ceiling4/4100%1.00247/247 successful

Wrong-evidence causal controlExact-release deterministic negative control; not a model rank.

Adversarial discrimination floor0/40%0.00247/247 successful

Frontier model submissionsAgent, model revision, trials, costs, and full traces required.

Model leaderboard———Awaiting first run

Why these controls matter

A 100% reference ceiling shows the tasks are executable, not all failing. A 0% wrong-evidence floor shows the verifier rejects a plausible workflow that relies on the wrong source record. Neither row measures model capability.

Task browser

Read every released task.

These are 4 independent Blobfish v3.1 realism tasks—not excerpts relabeled from the upstream suite. Each is a high-level employee request backed by a multi-source evidence room, three operating options, a task-specific causal evaluation narrative, and deterministic state checks.

train69 reference stepshard · generator label

bio_promote_hit_compound

Should Velatinib advance to confirmation before tomorrow's plate schedule is locked? The latest run contains a measurement the plate team does not trust, so the headline number may be misleading. Make a defensible potency call using only observations that are actually usable for this assay and be careful about the second compound record with the same common name. If the evidence clears the program threshold, record the hit and queue the controlled follow-up; otherwise hold it. Give the project lead the potency behind your decision.

How the employee outcome is evaluated

Evidence has to support the decision, the write, and the final answer.

Provider evidence: 26 exact reads
Live records: 34 correlated reads
Supported writes: 4
Proof: 4 readbacks

Reasoning contract

Call the potency of assay run 9301. Compute the mean ic50_nm across its replicates that PASSED QC (qc_flag 'pass') — replicates flagged 'fail' are excluded. A mean below 100 nM makes the compound a hit, so: set compound 9101's status to 'hit' and record the mean in its screening_note, set assay run 9301's status to 'complete', and queue confirmation by creating an assay run on assay 9201 for compound 9101 with run_on '2026-08-01', plate_label 'CONF-001' and status 'pending'.

Deterministic success

All exact provider and live-domain evidence is read before the first write; the task-specific predicates support the chosen branch; only the authorized state delta and one completion row are created; the persisted result is reopened.

Exact total call order is not graded. Equivalent investigation is accepted when the same task-specific evidence is established before mutation, the supported state is produced, and get_assay_run → get_assay_run → get_compound → get_task_result reopens every changed provider row and the persisted result.

Required evidence systems

01gmail.messages.list
02gmail.messages.get
03google_drive.files.list
04google_drive.files.get
05google_sheets.spreadsheets.values.get
06slack.search_messages
07slack.conversations_replies
08search_targets
09count_targets
10get_target
11search_compounds
12count_compounds
13get_compound
14search_assays
15count_assays
16get_assay
17search_assay_runs
18count_assay_runs
19get_assay_run
20search_assay_results
21count_assay_results
22get_assay_result
23search_literature_refs
24count_literature_refs
25get_literature_ref

Reference: PASS
Wrong-evidence floor: FAIL
Verifier: exact final tables, argument-aware causal trace, persisted result, and readback
Gradable: Yes

Operating options

✓
Complete the bounded supported outcomeCurrent authority, usable live evidence, and the available internal review window all support this task-scoped outcome.
02
Hold for an independent evidence reviewThis remains within authority but misses the documented need date even though the current evidence has no unresolved control conflict.
03
Extend remediation to neighboring recordsThe wider change exceeds both the approved immutable-record scope and the current incremental-cost authority.

Evidence files: 28
Exact criteria: 66
Allowed writes: 4
Call order: Not graded

Task evidenceTypeSizeSHA-256Open

operating-policy.mdassets/bio_promote_hit_compound/operating-policy.md

MD1.0 KB9cc6cf3e6530d8d5…View ↗

source-request.emlassets/bio_promote_hit_compound/source-request.eml

EML901 Bf0927f184ddbc033…View ↗

assay-method-and-literature.pdfassets/bio_promote_hit_compound/assay-method-and-literature.pdf

PDF941 B4bfa5eadd1f7bb0c…View ↗

replicate-qc-workbook.xlsxassets/bio_promote_hit_compound/replicate-qc-workbook.xlsx

XLSX8.8 KB4377c14cb9af66c2…View ↗

approval-decision.jsonassets/bio_promote_hit_compound/approval-decision.json

JSON804 Bbcd71ec4936c9eef…View ↗

team-slack-intake.jsonassets/bio_promote_hit_compound/team-slack-intake.json

JSON844 Bb6f25e3e7a871d64…View ↗

drive-source-index.jsonassets/bio_promote_hit_compound/drive-source-index.json

JSON394 B95e102b7237cf68b…View ↗

provider-contracts.jsonassets/bio_promote_hit_compound/provider-contracts.json

JSON23.6 KBa1a5a4165ede9c55…View ↗

former-owner-suggestion.emlassets/bio_promote_hit_compound/former-owner-suggestion.eml

EML627 B768e2e3a42a19361…View ↗

approval-scope.emlassets/bio_promote_hit_compound/approval-scope.eml

EML870 Bd3e45aa3895fe814…View ↗

operations-slack-thread.jsonassets/bio_promote_hit_compound/operations-slack-thread.json

JSON433 B7e8ebef6d43def44…View ↗

drive-approval-record.jsonassets/bio_promote_hit_compound/drive-approval-record.json

JSON628 Bbcdd946dbb228dbc…View ↗

authority-revision-2026-3.mdassets/bio_promote_hit_compound/authority-revision-2026-3.md

MD287 Bf739849733ac905b…View ↗

authority-revision-2025-4.mdassets/bio_promote_hit_compound/authority-revision-2025-4.md

MD227 B7cdbaa027866186b…View ↗

control-standard-2025-4.pdfassets/bio_promote_hit_compound/control-standard-2025-4.pdf

PDF823 Beab7bd11b6456a9c…View ↗

review-calendar.xlsxassets/bio_promote_hit_compound/review-calendar.xlsx

XLSX2.1 KBcdf5485e635a5b70…View ↗

record-lineage.csvassets/bio_promote_hit_compound/record-lineage.csv

CSV546 B29d60bbccae47832…View ↗

system-audit.logassets/bio_promote_hit_compound/system-audit.log

LOG882 Bd27a3d7988e73896…View ↗

live-snapshot.jsonassets/bio_promote_hit_compound/live-snapshot.json

JSON13.0 KB2a876e716729f8b8…View ↗

evidence-status.yamlassets/bio_promote_hit_compound/evidence-status.yaml

YAML230 B6a61fae6d7bdf230…View ↗

custodian-notes.txtassets/bio_promote_hit_compound/custodian-notes.txt

TXT212 B33a58a5b8030a994…View ↗

targets-system-export.csvassets/bio_promote_hit_compound/targets-system-export.csv

CSV2.4 KBa6aacf52a1abf7e9…View ↗

compounds-system-export.csvassets/bio_promote_hit_compound/compounds-system-export.csv

CSV2.4 KBbd0113cc6acc4bcf…View ↗

assays-system-export.csvassets/bio_promote_hit_compound/assays-system-export.csv

CSV2.2 KBa7b7b26163510e49…View ↗

assay_runs-system-export.csvassets/bio_promote_hit_compound/assay_runs-system-export.csv

CSV2.1 KBbfac17c29ae1c6a1…View ↗

assay_results-system-export.csvassets/bio_promote_hit_compound/assay_results-system-export.csv

CSV1.8 KBc9d2a79d734c77ac…View ↗

literature_refs-system-export.csvassets/bio_promote_hit_compound/literature_refs-system-export.csv

CSV1.8 KB46d470c251444dee…View ↗

asset-manifest.jsonassets/bio_promote_hit_compound/asset-manifest.json

JSON4.4 KBeacacc9192292876…View ↗

Inspect all 66 deterministic criteria

01
identity · identity_assay_resultsFor bio_promote_hit_compound, resolve assay_results by immutable result_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_assay_results"}, {"args": {}, "tool": "count_assay_results"}, {"args": {"limit": 50, "run_id": 9301}, "tool": "search_assay_results"}, {"args": {"run_id": 9301}, "tool": "count_assay_results"}, {"args": {"result_id": 9401}, "tool": "get_assay_result"}, {"args": {"result_id": 9402}, "tool": "get_assay_result"}, {"args": {"result_id": 9403}, "tool": "get_assay_result"}, {"args": {"result_id": 9404}, "tool": "get_assay_result"}].
02
investigation · investigation_assay_resultsEstablish the relevant and competing assay_results records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
03
identity · identity_assay_runsFor bio_promote_hit_compound, resolve assay_runs by immutable run_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_assay_runs"}, {"args": {}, "tool": "count_assay_runs"}, {"args": {"assay_id": 9201, "limit": 50}, "tool": "search_assay_runs"}, {"args": {"assay_id": 9201}, "tool": "count_assay_runs"}, {"args": {"run_id": 9301}, "tool": "get_assay_run"}].
04
investigation · investigation_assay_runsEstablish the relevant and competing assay_runs records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
05
identity · identity_assaysFor bio_promote_hit_compound, resolve assays by immutable assay_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_assays"}, {"args": {}, "tool": "count_assays"}, {"args": {"limit": 50, "target_id": 9001}, "tool": "search_assays"}, {"args": {"target_id": 9001}, "tool": "count_assays"}, {"args": {"assay_id": 9201}, "tool": "get_assay"}].
06
investigation · investigation_assaysEstablish the relevant and competing assays records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
07
identity · identity_compoundsFor bio_promote_hit_compound, resolve compounds by immutable compound_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_compounds"}, {"args": {}, "tool": "count_compounds"}, {"args": {"limit": 50, "name": "Velatinib"}, "tool": "search_compounds"}, {"args": {"name": "Velatinib"}, "tool": "count_compounds"}, {"args": {"compound_id": 9101}, "tool": "get_compound"}, {"args": {"compound_id": 9102}, "tool": "get_compound"}].
08
investigation · investigation_compoundsEstablish the relevant and competing compounds records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
09
identity · identity_literature_refsFor bio_promote_hit_compound, resolve literature_refs by immutable ref_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_literature_refs"}, {"args": {}, "tool": "count_literature_refs"}, {"args": {"limit": 50, "target_id": 9001}, "tool": "search_literature_refs"}, {"args": {"target_id": 9001}, "tool": "count_literature_refs"}, {"args": {"ref_id": 9501}, "tool": "get_literature_ref"}].
10
investigation · investigation_literature_refsEstablish the relevant and competing literature_refs records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
11
identity · identity_targetsFor bio_promote_hit_compound, resolve targets by immutable target_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_targets"}, {"args": {}, "tool": "count_targets"}, {"args": {"gene_symbol": "KRAS", "limit": 50}, "tool": "search_targets"}, {"args": {"gene_symbol": "KRAS"}, "tool": "count_targets"}, {"args": {"target_id": 9001}, "tool": "get_target"}].
12
investigation · investigation_targetsEstablish the relevant and competing targets records before deciding the bio_promote_hit_compound outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
13
investigation · investigation_provider_gmailFor bio_promote_hit_compound, use task-scoped gmail discovery and exact record reads to inspect source-request.eml (msg-5eba212daa12a921f8ba), former-owner-suggestion.eml (msg-596966b0742984e5f6b0), approval-scope.eml (msg-6c1fac51fcecbf2ee332). The remaining 0 listed gmail records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"maxResults": 100, "q": "\"DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND\"", "userId": "me"}, "tool": "gmail.messages.list"}, {"args": {"format": "raw", "id": "msg-5eba212daa12a921f8ba", "userId": "me"}, "tool": "gmail.messages.get"}, {"args": {"format": "raw", "id": "msg-596966b0742984e5f6b0", "userId": "me"}, "tool": "gmail.messages.get"}, {"args": {"format": "raw", "id": "msg-6c1fac51fcecbf2ee332", "userId": "me"}, "tool": "gmail.messages.get"}].
14
investigation · investigation_provider_google_driveFor bio_promote_hit_compound, use task-scoped google_drive discovery and exact record reads to inspect operating-policy.md (file-4d63be48184c7010ac13), assay-method-and-literature.pdf (file-3be5088958ab488b9c46), approval-decision.json (file-aa0caa876074cbdf5c40), drive-source-index.json (file-b7cf6fed9c197c809692), drive-approval-record.json (file-376ff5258b0cb3a126a7), authority-revision-2026-3.md (file-00fb9185980c00ea524b), authority-revision-2025-4.md (file-6652b3e631db1377490f), record-lineage.csv (file-ef89ce3461eb6d5ba840), live-snapshot.json (file-377f972f026d91169ff4), targets-system-export.csv (file-dc302e050fb24d92a865), compounds-system-export.csv (file-1f29f84c5ccedac88a7d), assays-system-export.csv (file-cfc72f1f79439298a154), assay_runs-system-export.csv (file-e51fe9b19d7368e3f5ba), assay_results-system-export.csv (file-d856c6205cc7bab48465), literature_refs-system-export.csv (file-7d110d16f47b6a8942b4), asset-manifest.json (file-d93b08e22c254c5b8c00). The remaining 5 listed google_drive records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"fields": "files(id,name,mimeType,modifiedTime,version)", "pageSize": 1000, "q": "fullText contains 'DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND'"}, "tool": "google_drive.files.list"}, {"args": {"alt": "media", "fileId": "file-4d63be48184c7010ac13"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-3be5088958ab488b9c46"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-aa0caa876074cbdf5c40"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-b7cf6fed9c197c809692"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-d856c6205cc7bab48465"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-e51fe9b19d7368e3f5ba"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-cfc72f1f79439298a154"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-1f29f84c5ccedac88a7d"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-7d110d16f47b6a8942b4"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-dc302e050fb24d92a865"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-376ff5258b0cb3a126a7"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-00fb9185980c00ea524b"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-6652b3e631db1377490f"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-ef89ce3461eb6d5ba840"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-377f972f026d91169ff4"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-d93b08e22c254c5b8c00"}, "tool": "google_drive.files.get"}].
15
investigation · investigation_provider_google_sheetsFor bio_promote_hit_compound, use task-scoped google_sheets discovery and exact record reads to inspect replicate-qc-workbook.xlsx (sheet-789b97e3ffca87aed4a8), review-calendar.xlsx (sheet-5e6e695794335b85fdcd). The remaining 0 listed google_sheets records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"range": "A:Z", "spreadsheetId": "sheet-789b97e3ffca87aed4a8"}, "tool": "google_sheets.spreadsheets.values.get"}, {"args": {"range": "A:Z", "spreadsheetId": "sheet-5e6e695794335b85fdcd"}, "tool": "google_sheets.spreadsheets.values.get"}].
16
investigation · investigation_provider_slackFor bio_promote_hit_compound, use task-scoped slack discovery and exact record reads to inspect team-slack-intake.json (thread-5bed1abb32882629bc1d), operations-slack-thread.json (thread-6621e02919f70c1a411c). The remaining 0 listed slack records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"count": 100, "query": "\"DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND\"", "sort": "timestamp", "sort_dir": "asc"}, "tool": "slack.search_messages"}, {"args": {"channel": "#assay-review", "limit": 100, "ts": "1724937600.864773"}, "tool": "slack.conversations_replies"}, {"args": {"channel": "#assay-review", "limit": 100, "ts": "1724937600.359038"}, "tool": "slack.conversations_replies"}].
17
authority · authority_effective_revisionUse the effective 2026.3 operating control for bio_promote_hit_compound, corroborated by the approval and Drive revision records. Enforced by current-control PDF, Drive record, and policy revision identity.
18
authority · authority_retired_conflictTreat the retained 2025.4 note for bio_promote_hit_compound as a competing historical hypothesis, not as permission or current authority. Enforced by current-versus-retired revision comparison.
19
correlation · correlation_cross_system_identityCorrelate the employee request, controlled workbook, system exports, Slack discussion, Drive record, and approval email for bio_promote_hit_compound by case and immutable record IDs. Enforced by cross-file case IDs and exact world selectors.
20
correlation · correlation_live_over_snapshotReconcile the released snapshot for bio_promote_hit_compound to successful live MCP reads before relying on either source; the files alone do not establish the final outcome. Enforced by asset digests plus exact live trace.
21
correlation · correlation_count_detail_relationshipFor bio_promote_hit_compound, reconcile broad counts with detail records and foreign-key relationships so duplicates and similarly named distractors are not collapsed. Enforced by search/count/get selector intersection.
22
identity · identity_asset_lineageBind every cited bio_promote_hit_compound file to its case-specific asset-manifest digest and reject evidence copied from another task, even when its filename and business vocabulary look plausible. Enforced by globally unique asset SHA-256 census and manifest membership.
23
investigation · investigation_structured_exportsReconcile the bio_promote_hit_compound structured exports (assay_results-system-export.csv, assay_runs-system-export.csv, assays-system-export.csv, compounds-system-export.csv, literature_refs-system-export.csv, targets-system-export.csv) with the live records they represent; no single export is sufficient when related tables disagree. Enforced by exact table selectors, live-state snapshot, and parser-validated CSV records.
24
investigation · investigation_controlled_workbookUse the parser-validated review-calendar workbook for bio_promote_hit_compound to test timing and capacity assumptions against current records, while treating workbook conclusions as evidence rather than authorization. Enforced by XLSX parser gate, workbook digest, and exact temporal selectors.
25
correlation · correlation_communication_authority_splitFor bio_promote_hit_compound, distinguish what the source request, operations Slack thread, and former-owner message say from what the signed approval and current control actually authorize. Enforced by communication-record correlation plus current-authority and approval-scope checks.
26
correlation · correlation_provider_live_lineageJoin the task-scoped Gmail, Drive, Sheets, and Slack records for bio_promote_hit_compound to the exact live-domain selectors by case id, provider id, immutable row id, and effective revision; a same-name file or row from another task is not equivalent evidence. Enforced by provider asset ids and SHA-256 manifest joined to argument-aware domain selectors.
27
alternatives · alternatives_blast_radiusCompare the complete table-level blast radius of the supported bio_promote_hit_compound branch with the hold and broad-cleanup alternatives; reject any option that changes an unapproved neighbor even if its headline answer is correct. Enforced by all-table before/after digests and unauthorized-write control.
28
decision · decision_citation_lineageCarry the selected bio_promote_hit_compound decision into the result row with the exact current evidence IDs used to derive it, so a correct-looking answer backed by stale or cross-task evidence fails. Enforced by exact result evidence set and wrong-evidence control.
29
alternatives · alternatives_supportedKeep the bounded evidence-supported outcome for bio_promote_hit_compound available only if effective authority and current records agree. Enforced by exact final-state and result contract.
30
alternatives · alternatives_holdRetain a hold/escalate branch for bio_promote_hit_compound if controlling records conflict or authority is absent; difficulty alone is not a reason to hold. Enforced by wrong-decision negative control.
31
alternatives · alternatives_broadReject a broad neighboring-record cleanup for bio_promote_hit_compound, even when similar names or stale notes make it superficially plausible. Enforced by unauthorized-write negative control.
32
alternatives · alternatives_no_changeEvaluate a no-domain-change outcome for bio_promote_hit_compound: retain it only when the exact live predicates do not authorize a mutation; otherwise it is an incomplete state-only or shortcut branch. Enforced by task-specific source VCode plus no-op and state-only controls.
33
decision · decision_supported_branchDerive the selected bio_promote_hit_compound branch from the intersection of authority, identity, relationship, timing, and live-state evidence—not from the prompt's requested outcome. Enforced by source VCode, exact state digests, and task-result row.
34
decision · decision_task_specific_causal_pathThe deterministic reasoning for bio_promote_hit_compound is task-specific: Call the potency of assay run 9301. Compute the mean ic50_nm across its replicates that PASSED QC (qc_flag 'pass') — replicates flagged 'fail' are excluded. A mean below 100 nM makes the compound a hit, so: set compound 9101's status to 'hit' and record the mean in its screening_note, set assay run 9301's status to 'complete', and queue confirmation by creating an assay run on assay 9201 for compound 9101 with run_on '2026-08-01', plate_label 'CONF-001' and status 'pending'. The model must establish each predicate from the provider evidence and live domain records, then carry only the supported branch into state. Enforced by exact source VCode, authored selectors, before/after state, and adversarial branch controls.
35
procedure · procedure_successful_evidenceEvery exact contextual selector required for bio_promote_hit_compound must succeed; a failed or differently filtered call is not equivalent evidence. Enforced by argument-aware successful trace.
36
procedure · procedure_provider_discovery_before_detailFor bio_promote_hit_compound, complete the task-scoped Gmail, Drive, and Slack discovery calls before relying on exact message, file, spreadsheet, or thread records; the verifier accepts equivalent causal ordering but requires every exact discovery and detail selector before state changes. Enforced by provider list/search and exact-get trace selectors before first mutation.
37
procedure · procedure_read_before_writeComplete all 60 causal reads for bio_promote_hit_compound before the first domain mutation or persisted completion result. Enforced by trace sequence bounds.
38
procedure · procedure_result_after_workPersist the exact bio_promote_hit_compound result only after any supported domain transition has completed. Enforced by source-mutation-before-submit sequence check.
39
procedure · procedure_post_write_readbackReopen the exact persisted bio_promote_hit_compound result after submission so verification observes stored state rather than trusting the write response. Enforced by exact get_task_result selector after submit.
40
state · state_assay_runsProduce only the supported bio_promote_hit_compound transition in assay_runs: added run_id=['9303']; removed run_id=none; updated ["run_id=9301 (status:'pending'→'complete')"]. The complete final table must match digest b72d6d91a248a2fa4fbcbfbe09b4e71cb02eb16965adadd9690b2b33f6e051b1. Enforced by exact complete-table digest and source VCode.
41
state · state_compoundsProduce only the supported bio_promote_hit_compound transition in compounds: added compound_id=none; removed compound_id=none; updated ["compound_id=9101 (screening_note:''→'Mean QC-passing IC50 22 nM on run 9301.', status:'screening'→'hit')"]. The complete final table must match digest 15de047a0c7f49f7e03f5c67986ecbc10e575dbad9fd5ecc1a1c40126f553c16. Enforced by exact complete-table digest and source VCode.
42
state · state_authorized_transition_01The bio_promote_hit_compound state transition with exact arguments {"compound_id": 9101, "screening_note": "Mean QC-passing IC50 22 nM on run 9301.", "status": "hit"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
43
state · state_authorized_transition_02The bio_promote_hit_compound state transition with exact arguments {"run_id": 9301, "status": "complete"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
44
state · state_authorized_transition_03The bio_promote_hit_compound state transition with exact arguments {"assay_id": 9201, "compound_id": 9101, "plate_label": "CONF-001", "run_on": "2026-08-01", "status": "pending"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
45
state · state_authorized_transition_04The bio_promote_hit_compound state transition with exact arguments {"channel": "#assay-review", "client_msg_id": "b6cc4f06e42e4cc0a6cc3aaadf06457a", "text": "Record compound 9101 as a 22 nM hit, complete run 9301, and queue confirmation plate CONF-001. I reconciled 119 candidate rows across 6 live record families, excluded 109 neighbor rows, and retained 10 immutable task records. The selected outcome is scheduled for 2026-09-03 (-1 days versus the 2026-09-04 need date), costs $2600, fits with 6 internal hours to spare, and is authorized by APR-4fb63d599e4dffb4. The broader neighbor remediation requires additional approval and was not executed. Exact answer: 22."} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
46
state · state_source_outcomeAll 2 task-specific executable outcome assertions for bio_promote_hit_compound must hold against the final sandbox state. Enforced by checked-in deterministic VCode.
47
state · state_result_rowCreate exactly one task-result row for bio_promote_hit_compound containing the supported decision, exact summary, evidence IDs, causal workpaper, and exact answer when applicable. Enforced by persisted task_results state.
48
answer · answer_result_submittedReturn an audit-ready bio_promote_hit_compound conclusion in the persisted result row rather than only narrating intended work. Enforced by exact task-result state verifier.
49
answer · answer_evidence_citationsCite the current policy, source request, controlled workbook, and effective source document for bio_promote_hit_compound; stale authority cannot substitute. Enforced by exact required evidence set.
50
answer · answer_exact_valueReturn the exact task-supported scalar for bio_promote_hit_compound, derived from the live records rather than copied from an asset. Enforced by deterministic submitted-answer equality.
51
containment · containment_write_scopeUse only the bio_promote_hit_compound authorized domain mutation set and the task-result handoff; no neighboring table or record may change. Enforced by allowed-write trace plus all-table digests.
52
containment · containment_no_rejected_mutationNo state-changing request in bio_promote_hit_compound may fail and then be hidden by a later correct-looking final answer. Enforced by successful mutation trace.
53
identity · identity_H01_scope_identityLocate bio_promote_hit_compound through case DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND and the exact immutable target map {"assay_results": [9401, 9402, 9403, 9404], "assay_runs": [9301], "assays": [9201], "compounds": [9101, 9102], "literature_refs": [9501], "targets": [9001]}. Enforced by exact workpaper.scope plus successful argument-aware search/count/get evidence.
54
decision · decision_H02_requirement_derivationApply the controlled task rule from operating-policy.md#task-specific-decision-control and derive all 6 required record families before choosing an outcome. Enforced by exact workpaper.requirement and D02.
55
investigation · investigation_H03_availability_reconciliationReconcile 119 observed candidate rows, 109 excluded neighbors, and 10 usable targets. Enforced by exact workpaper.reconciliation and D03-D05.
56
decision · decision_H04_gap_or_boundNet 6 required families against 6 covered families into exact gap 0. Enforced by exact workpaper.gapOrBound and D06.
57
authority · authority_H05_external_constraintApply independent approval APR-4fb63d599e4dffb4 and its $2800 cost boundary. Enforced by exact workpaper.externalConstraint corroborated by approval email and Drive record.
58
investigation · investigation_H06_internal_constraintApply the 2026-09-04 business need, 19 available hours, and $200 hourly rate from the controlled calendar. Enforced by exact workpaper.internalConstraint and D07-D10.
59
alternatives · alternatives_H07_qualified_optionsGrade all three task-specific alternatives with exact outcome, date, signed variance, incremental cost, authority status, and one selected branch. Enforced by exact workpaper.options equality and wrong-reasoning control.
60
decision · decision_H08_recommendationSelect evidence-backed-scoped-outcome with exact outcome 'Record compound 9101 as a 22 nM hit, complete run 9301, and queue confirmation plate CONF-001.'. Enforced by exact workpaper.recommendation and D13.
61
decision · decision_H09_control_comparisonCompare outcome 2026-09-03 to need 2026-09-04 into signed variance -1 and status ON_OR_BEFORE_NEED. Enforced by independently recomputed exact workpaper.controlComparison and D11.
62
authority · authority_H10_authority_applicationApply authority APR-4fb63d599e4dffb4 to the selected immutable scope and reject broad-neighbor-remediation. Enforced by exact workpaper.authorityApplication and D12.
63
state · state_H11_controlled_executionExecute exactly 3 authorized domain writes across 2 changed tables, perform 3 exact-record provider readbacks, then reopen the persisted result. Enforced by exact writes, complete-table digests, provider-native exact-record readbacks, persisted result, and result readback trace.
64
answer · answer_H12_stakeholder_communicationPost the exact stakeholder handoff to #assay-review carrying the selected outcome, timing, cost, capacity constraint, and authority. Enforced by exact slack.chat_postMessage call and persisted stakeholder-message row.
65
answer · answer_H13_exact_intermediatesPersist all 13 dependent derivations and every intermediate requirement, reconciliation, constraint, option, authority, state, and answer field exactly. Enforced by exact causal-workpaper equality; wrong-reasoning negative control.
66
containment · containment_visible_hidden_partitionThe 28 files visible for bio_promote_hit_compound contain evidence only; verifier digests, the oracle trajectory, and precomputed outcomes remain outside the asset room. Enforced by release-tree partition and leakage gate.

Generator label only; no empirical model difficulty claim is published.

Episode trace

Follow the tools, outputs, and verdict.

Switch between a successful reference replay and a real failing floor episode. Both start from the same immutable world state; neither is a model leaderboard row.

deterministic solvability ceiling

Reference solver

A checked-in v3 causal investigation replayed against a fresh SQLite copy. This proves the task can pass; it is not a model submission.

Task: bio_promote_hit_compound
Steps: 69
Reward: 1.0
Status: passed

00
Task loadeduser → agent
Fresh seeded database copied into an isolated episode.

gmail.messages.listtool success

Arguments

{
  "maxResults": 100,
  "q": "\"DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND\"",
  "userId": "me"
}

Result

{
  "messages": [
    {
      "id": "msg-5eba212daa12a921f8ba",
      "threadId": "msg-5eba212daa12a921f8ba"
    },
    {
      "id": "msg-596966b0742984e5f6b0",
      "threadId": "msg-596966b0742984e5f6b0"
    },
    {
      "id": "msg-6c1fac51fcecbf2ee332",
      "threadId": "msg-6c1fac51fcecbf2ee332"
    }
  ],
  "resultSizeEstimate": 3
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-5eba212daa12a921f8ba",
  "userId": "me"
}

Result

{
  "id": "msg-5eba212daa12a921f8ba",
  "name": "source-request.eml",
  "mimeType": "message/rfc822",
  "sha256": "f0927f184ddbc03365cc3990520c05486ab45344527f7bee5c283fb36973b851",
  "bytes": 901
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-596966b0742984e5f6b0",
  "userId": "me"
}

Result

{
  "id": "msg-596966b0742984e5f6b0",
  "name": "former-owner-suggestion.eml",
  "mimeType": "message/rfc822",
  "sha256": "768e2e3a42a19361fd0c07f9a96aeea2a1bc240c3423ecd75284729c112d74b0",
  "bytes": 627
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-6c1fac51fcecbf2ee332",
  "userId": "me"
}

Result

{
  "id": "msg-6c1fac51fcecbf2ee332",
  "name": "approval-scope.eml",
  "mimeType": "message/rfc822",
  "sha256": "d3e45aa3895fe81438b3538046f8a81b9af3addf63c7d1761481050b581d148b",
  "bytes": 870
}

google_drive.files.listtool success

Arguments

{
  "fields": "files(id,name,mimeType,modifiedTime,version)",
  "pageSize": 1000,
  "q": "fullText contains 'DB24-DISCOVERYBENCH-102-BIO_PROMOTE_HIT_COMPOUND'"
}

Result

{
  "files": [
    {
      "id": "file-4d63be48184c7010ac13",
      "name": "operating-policy.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-3be5088958ab488b9c46",
      "name": "assay-method-and-literature.pdf",
      "mimeType": "application/pdf",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "sheet-789b97e3ffca87aed4a8",
      "name": "replicate-qc-workbook.xlsx",
      "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-aa0caa876074cbdf5c40",
      "name": "approval-decision.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-b7cf6fed9c197c809692",
      "name": "drive-source-index.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-b8e671bcc8dcd3463aac",
      "name": "provider-contracts.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-376ff5258b0cb3a126a7",
      "name": "drive-approval-record.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-00fb9185980c00ea524b",
      "name": "authority-revision-2026-3.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-6652b3e631db1377490f",
      "name": "authority-revision-2025-4.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-41dde1e119a99c58871b",
      "name": "control-standard-2025-4.pdf",
      "mimeType": "application/pdf",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "sheet-5e6e695794335b85fdcd",
      "name": "review-calendar.xlsx",
      "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-ef89ce3461eb6d5ba840",
      "name": "record-lineage.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-8e7246ff6d1aaf95ecc3",
      "name": "system-audit.log",
      "mimeType": "text/plain",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-377f972f026d91169ff4",
      "name": "live-snapshot.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-47da6e96f80398ec3f22",
      "name": "evidence-status.yaml",
      "mimeType": "application/yaml",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-8cee22eac81048ab9bd1",
      "name": "custodian-notes.txt",
      "mimeType": "text/plain",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-dc302e050fb24d92a865",
      "name": "targets-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-1f29f84c5ccedac88a7d",
      "name": "compounds-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-cfc72f1f79439298a154",
      "name": "assays-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-e51fe9b19d7368e3f5ba",
      "name": "assay_runs-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-d856c6205cc7bab48465",
      "name": "assay_results-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-7d110d16f47b6a8942b4",
      "name": "literature_refs-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-d93b08e22c254c5b8c00",
      "name": "asset-manifest.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    }
  ]
}