All benchmarksFinancial research

FilingBench-50Evidence 2026-08-29Independent companionDataset page live

Can an agent find the number, show its provenance, and update the model?

Public financial-research questions paired with a deterministic DCF, assumptions, period-data, and review-note world.

Explore 4 companion tasks ↓World repository ↗Run on Harbor ↗

50upstream Harbor tasks

4independent companion tasks

28files per task room

56+criteria per task

Baseline leaderboard

Solvable, discriminating, and honest about what ran.

All companion tasks are solvable by their reference traces, exact replays, and reject thirteen task-applicable controls including correct state with wrong reasoning, wrong target, and keyword stuffing. No paid or frontier model run is claimed. These 4 tasks are Blobfish-authored companions, not conversions of the 50-task upstream suite. The first two rows are harness controls; the model leaderboard remains open until a version-pinned submission exists.

EntryRoleTasksPass rateMean rewardTool calls

Reference solverDeterministic trace replay; not a model rank.

Solvability ceiling4/4100%1.00213/213 successful

Wrong-evidence causal controlExact-release deterministic negative control; not a model rank.

Adversarial discrimination floor0/40%0.00213/213 successful

Frontier model submissionsAgent, model revision, trials, costs, and full traces required.

Model leaderboard———Awaiting first run

Why these controls matter

A 100% reference ceiling shows the tasks are executable, not all failing. A 0% wrong-evidence floor shows the verifier rejects a plausible workflow that relies on the wrong source record. Neither row measures model capability.

Task browser

Read every released task.

These are 4 independent Blobfish v3.1 realism tasks—not excerpts relabeled from the upstream suite. Each is a high-level employee request backed by a multi-source evidence room, three operating options, a task-specific causal evaluation narrative, and deterministic state checks.

train61 reference stepshard · generator label

fa_finalize_solstice_dcf

Can you tell me whether the Solstice Energy base case is actually ready for this afternoon's review? The number circulating has not been signed off, and I do not want us finalizing a model built on the wrong issuer history or an unsupported assumption. If the evidence holds, finish the controlled review and give the Head of Research the enterprise value and a short audit trail. If it does not, leave the model under review and explain what blocks it.

How the employee outcome is evaluated

Evidence has to support the decision, the write, and the final answer.

Provider evidence: 26 exact reads
Live records: 26 correlated reads
Supported writes: 4
Proof: 4 readbacks

Reasoning contract

Finalise the Solstice base-case DCF (model_id 9201, company 9001). Total the free_cash_flow across every financial period belonging to company 9001 — other companies' periods do not count. Set model 9201's enterprise_value to that total and its status to 'final', mark its WACC assumption (assumption_id 9301) as status 'verified', and file the review: create a review note on model 9201 from reviewer 'Head of Research' with status 'open' and a body containing the summed cash flow.

Deterministic success

All exact provider and live-domain evidence is read before the first write; the task-specific predicates support the chosen branch; only the authorized state delta and one completion row are created; the persisted result is reopened.

Exact total call order is not graded. Equivalent investigation is accepted when the same task-specific evidence is established before mutation, the supported state is produced, and get_dcf_assumption → get_dcf_model → get_review_note → get_task_result reopens every changed provider row and the persisted result.

Required evidence systems

01gmail.messages.list
02gmail.messages.get
03google_drive.files.list
04google_drive.files.get
05google_sheets.spreadsheets.values.get
06slack.search_messages
07slack.conversations_replies
08search_fa_companies
09count_fa_companies
10get_fa_company
11search_dcf_models
12count_dcf_models
13get_dcf_model
14search_financial_periods
15count_financial_periods
16get_financial_period
17search_dcf_assumptions
18count_dcf_assumptions
19get_dcf_assumption
20search_review_notes
21count_review_notes

Reference: PASS
Wrong-evidence floor: FAIL
Verifier: exact final tables, argument-aware causal trace, persisted result, and readback
Gradable: Yes

Operating options

✓
Complete the bounded supported outcomeCurrent authority, usable live evidence, and the available internal review window all support this task-scoped outcome.
02
Hold for an independent evidence reviewThis remains within authority but misses the documented need date even though the current evidence has no unresolved control conflict.
03
Extend remediation to neighboring recordsThe wider change exceeds both the approved immutable-record scope and the current incremental-cost authority.

Evidence files: 28
Exact criteria: 65
Allowed writes: 4
Call order: Not graded

Task evidenceTypeSizeSHA-256Open

operating-policy.mdassets/fa_finalize_solstice_dcf/operating-policy.md

MD1.0 KB647e4314b566673f…View ↗

source-request.emlassets/fa_finalize_solstice_dcf/source-request.eml

EML813 Bdcd4775007ce0c1d…View ↗

issuer-filing-extract.pdfassets/fa_finalize_solstice_dcf/issuer-filing-extract.pdf

PDF941 Bc18635e93faddc01…View ↗

valuation-review-model.xlsxassets/fa_finalize_solstice_dcf/valuation-review-model.xlsx

XLSX9.4 KB3ea52cb056147504…View ↗

approval-decision.jsonassets/fa_finalize_solstice_dcf/approval-decision.json

JSON732 B579f3ad9dbc072a6…View ↗

team-slack-intake.jsonassets/fa_finalize_solstice_dcf/team-slack-intake.json

JSON759 Baea2b8306e5fc497…View ↗

drive-source-index.jsonassets/fa_finalize_solstice_dcf/drive-source-index.json

JSON373 B9e77066219a64f9d…View ↗

provider-contracts.jsonassets/fa_finalize_solstice_dcf/provider-contracts.json

JSON21.2 KB0ac6f1371d259bbf…View ↗

former-owner-suggestion.emlassets/fa_finalize_solstice_dcf/former-owner-suggestion.eml

EML635 B83a7a3d6c45695af…View ↗

approval-scope.emlassets/fa_finalize_solstice_dcf/approval-scope.eml

EML835 B167a45ac06457fda…View ↗

operations-slack-thread.jsonassets/fa_finalize_solstice_dcf/operations-slack-thread.json

JSON424 B02a0ebe66f7fe0dc…View ↗

drive-approval-record.jsonassets/fa_finalize_solstice_dcf/drive-approval-record.json

JSON558 B0b0d71b71bc7d991…View ↗

authority-revision-2026-3.mdassets/fa_finalize_solstice_dcf/authority-revision-2026-3.md

MD275 Bbb1e6ea5d3767f1d…View ↗

authority-revision-2025-4.mdassets/fa_finalize_solstice_dcf/authority-revision-2025-4.md

MD219 Bdcb5c1a8c1daa814…View ↗

control-standard-2025-4.pdfassets/fa_finalize_solstice_dcf/control-standard-2025-4.pdf

PDF815 B0322f2b21e691626…View ↗

review-calendar.xlsxassets/fa_finalize_solstice_dcf/review-calendar.xlsx

XLSX2.1 KB3844083dd8b4d80b…View ↗

record-lineage.csvassets/fa_finalize_solstice_dcf/record-lineage.csv

CSV522 B5fc65a9f4ce478c9…View ↗

system-audit.logassets/fa_finalize_solstice_dcf/system-audit.log

LOG874 B5803fc4b63dd7df6…View ↗

live-snapshot.jsonassets/fa_finalize_solstice_dcf/live-snapshot.json

JSON14.0 KBb7e80a4e1d90642f…View ↗

evidence-status.yamlassets/fa_finalize_solstice_dcf/evidence-status.yaml

YAML218 Bb4c6e2d4fc75a426…View ↗

custodian-notes.txtassets/fa_finalize_solstice_dcf/custodian-notes.txt

TXT208 B9298f8237444cb8f…View ↗

fa_companies-system-export.csvassets/fa_finalize_solstice_dcf/fa_companies-system-export.csv

CSV2.1 KBc6869242d6b34cd0…View ↗

dcf_models-system-export.csvassets/fa_finalize_solstice_dcf/dcf_models-system-export.csv

CSV2.0 KB1eca099dbd230603…View ↗

financial_periods-system-export.csvassets/fa_finalize_solstice_dcf/financial_periods-system-export.csv

CSV2.2 KBc4b614cda6aec610…View ↗

dcf_assumptions-system-export.csvassets/fa_finalize_solstice_dcf/dcf_assumptions-system-export.csv

CSV2.3 KB9da7d74f7894f79c…View ↗

review_notes-system-export.csvassets/fa_finalize_solstice_dcf/review_notes-system-export.csv

CSV4.4 KB9c62581579fe830d…View ↗

lbo_models-system-export.csvassets/fa_finalize_solstice_dcf/lbo_models-system-export.csv

CSV1.6 KB3cc8d7edc8abaabc…View ↗

asset-manifest.jsonassets/fa_finalize_solstice_dcf/asset-manifest.json

JSON4.4 KBda1c9994b623895b…View ↗

Inspect all 65 deterministic criteria

01
identity · identity_dcf_assumptionsFor fa_finalize_solstice_dcf, resolve dcf_assumptions by immutable assumption_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_dcf_assumptions"}, {"args": {}, "tool": "count_dcf_assumptions"}, {"args": {"limit": 50, "model_id": 9201}, "tool": "search_dcf_assumptions"}, {"args": {"model_id": 9201}, "tool": "count_dcf_assumptions"}, {"args": {"assumption_id": 9301}, "tool": "get_dcf_assumption"}].
02
investigation · investigation_dcf_assumptionsEstablish the relevant and competing dcf_assumptions records before deciding the fa_finalize_solstice_dcf outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
03
identity · identity_dcf_modelsFor fa_finalize_solstice_dcf, resolve dcf_models by immutable model_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_dcf_models"}, {"args": {}, "tool": "count_dcf_models"}, {"args": {"company_id": 9001, "limit": 50}, "tool": "search_dcf_models"}, {"args": {"company_id": 9001}, "tool": "count_dcf_models"}, {"args": {"model_id": 9201}, "tool": "get_dcf_model"}].
04
investigation · investigation_dcf_modelsEstablish the relevant and competing dcf_models records before deciding the fa_finalize_solstice_dcf outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
05
identity · identity_fa_companiesFor fa_finalize_solstice_dcf, resolve fa_companies by immutable company_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_fa_companies"}, {"args": {}, "tool": "count_fa_companies"}, {"args": {"limit": 50, "name": "Solstice Energy"}, "tool": "search_fa_companies"}, {"args": {"name": "Solstice Energy"}, "tool": "count_fa_companies"}, {"args": {"company_id": 9001}, "tool": "get_fa_company"}].
06
investigation · investigation_fa_companiesEstablish the relevant and competing fa_companies records before deciding the fa_finalize_solstice_dcf outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
07
identity · identity_financial_periodsFor fa_finalize_solstice_dcf, resolve financial_periods by immutable period_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_financial_periods"}, {"args": {}, "tool": "count_financial_periods"}, {"args": {"company_id": 9001, "limit": 50}, "tool": "search_financial_periods"}, {"args": {"company_id": 9001}, "tool": "count_financial_periods"}, {"args": {"period_id": 9101}, "tool": "get_financial_period"}, {"args": {"period_id": 9102}, "tool": "get_financial_period"}, {"args": {"period_id": 9103}, "tool": "get_financial_period"}].
08
investigation · investigation_financial_periodsEstablish the relevant and competing financial_periods records before deciding the fa_finalize_solstice_dcf outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
09
identity · identity_review_notesFor fa_finalize_solstice_dcf, resolve review_notes by immutable note_id and explicit relationships; matching display text is insufficient. Enforced by exact argument-aware selectors: [{"args": {"limit": 50}, "tool": "search_review_notes"}, {"args": {}, "tool": "count_review_notes"}, {"args": {"limit": 50, "model_id": 9201}, "tool": "search_review_notes"}, {"args": {"model_id": 9201}, "tool": "count_review_notes"}].
10
investigation · investigation_review_notesEstablish the relevant and competing review_notes records before deciding the fa_finalize_solstice_dcf outcome; broad search, count, and detail evidence must agree. Enforced by successful exact MCP requests plus isolated seed.
11
investigation · investigation_provider_gmailFor fa_finalize_solstice_dcf, use task-scoped gmail discovery and exact record reads to inspect source-request.eml (msg-6a02c41ec2c7908f56f9), former-owner-suggestion.eml (msg-a3859fdd8b6dde1dfd91), approval-scope.eml (msg-369888fa4cd226854a04). The remaining 0 listed gmail records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"maxResults": 100, "q": "\"DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF\"", "userId": "me"}, "tool": "gmail.messages.list"}, {"args": {"format": "raw", "id": "msg-6a02c41ec2c7908f56f9", "userId": "me"}, "tool": "gmail.messages.get"}, {"args": {"format": "raw", "id": "msg-a3859fdd8b6dde1dfd91", "userId": "me"}, "tool": "gmail.messages.get"}, {"args": {"format": "raw", "id": "msg-369888fa4cd226854a04", "userId": "me"}, "tool": "gmail.messages.get"}].
12
investigation · investigation_provider_google_driveFor fa_finalize_solstice_dcf, use task-scoped google_drive discovery and exact record reads to inspect operating-policy.md (file-4f7f602567f25f0ceced), issuer-filing-extract.pdf (file-85daaaa1f655f828efdb), approval-decision.json (file-ba1befb3f34aacd98f87), drive-source-index.json (file-449aa4d3f2962992baca), drive-approval-record.json (file-c17b25ec14d4ce70c9de), authority-revision-2026-3.md (file-76d1aff002da28751d67), authority-revision-2025-4.md (file-2a857ed446abda210217), record-lineage.csv (file-f4e565b8ccc3b9f169d6), live-snapshot.json (file-58f10bcdad11474ca4c3), fa_companies-system-export.csv (file-0a1551c275585b2dc5ed), dcf_models-system-export.csv (file-be74ae297db504cbda48), financial_periods-system-export.csv (file-f725914ea935e75b3d25), dcf_assumptions-system-export.csv (file-685a07277edb13d22e96), review_notes-system-export.csv (file-c307c47e0a4aca47b561), lbo_models-system-export.csv (file-ae994e57f782cf6f7a89), asset-manifest.json (file-58f8fe6ef69d7e523545). The remaining 5 listed google_drive records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"fields": "files(id,name,mimeType,modifiedTime,version)", "pageSize": 1000, "q": "fullText contains 'DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF'"}, "tool": "google_drive.files.list"}, {"args": {"alt": "media", "fileId": "file-4f7f602567f25f0ceced"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-85daaaa1f655f828efdb"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-ba1befb3f34aacd98f87"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-449aa4d3f2962992baca"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-685a07277edb13d22e96"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-be74ae297db504cbda48"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-0a1551c275585b2dc5ed"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-f725914ea935e75b3d25"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-ae994e57f782cf6f7a89"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-c307c47e0a4aca47b561"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-c17b25ec14d4ce70c9de"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-76d1aff002da28751d67"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-2a857ed446abda210217"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-f4e565b8ccc3b9f169d6"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-58f10bcdad11474ca4c3"}, "tool": "google_drive.files.get"}, {"args": {"alt": "media", "fileId": "file-58f8fe6ef69d7e523545"}, "tool": "google_drive.files.get"}].
13
investigation · investigation_provider_google_sheetsFor fa_finalize_solstice_dcf, use task-scoped google_sheets discovery and exact record reads to inspect valuation-review-model.xlsx (sheet-6754468a021f983acee3), review-calendar.xlsx (sheet-1d5a4ee07edbe874c9be). The remaining 0 listed google_sheets records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"range": "A:Z", "spreadsheetId": "sheet-6754468a021f983acee3"}, "tool": "google_sheets.spreadsheets.values.get"}, {"args": {"range": "A:Z", "spreadsheetId": "sheet-1d5a4ee07edbe874c9be"}, "tool": "google_sheets.spreadsheets.values.get"}].
14
investigation · investigation_provider_slackFor fa_finalize_solstice_dcf, use task-scoped slack discovery and exact record reads to inspect team-slack-intake.json (thread-de497d3e2d822bdb6034), operations-slack-thread.json (thread-943a3ffa7947f7161bd9). The remaining 0 listed slack records remain available for open-ended investigation but are not falsely claimed as inspected. The communication or file content is evidence; it does not precompute the business outcome. Enforced by exact documented provider selectors: [{"args": {"count": 100, "query": "\"DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF\"", "sort": "timestamp", "sort_dir": "asc"}, "tool": "slack.search_messages"}, {"args": {"channel": "#research-review", "limit": 100, "ts": "1724937600.330729"}, "tool": "slack.conversations_replies"}, {"args": {"channel": "#research-review", "limit": 100, "ts": "1724937600.967773"}, "tool": "slack.conversations_replies"}].
15
authority · authority_effective_revisionUse the effective 2026.3 operating control for fa_finalize_solstice_dcf, corroborated by the approval and Drive revision records. Enforced by current-control PDF, Drive record, and policy revision identity.
16
authority · authority_retired_conflictTreat the retained 2025.4 note for fa_finalize_solstice_dcf as a competing historical hypothesis, not as permission or current authority. Enforced by current-versus-retired revision comparison.
17
correlation · correlation_cross_system_identityCorrelate the employee request, controlled workbook, system exports, Slack discussion, Drive record, and approval email for fa_finalize_solstice_dcf by case and immutable record IDs. Enforced by cross-file case IDs and exact world selectors.
18
correlation · correlation_live_over_snapshotReconcile the released snapshot for fa_finalize_solstice_dcf to successful live MCP reads before relying on either source; the files alone do not establish the final outcome. Enforced by asset digests plus exact live trace.
19
correlation · correlation_count_detail_relationshipFor fa_finalize_solstice_dcf, reconcile broad counts with detail records and foreign-key relationships so duplicates and similarly named distractors are not collapsed. Enforced by search/count/get selector intersection.
20
identity · identity_asset_lineageBind every cited fa_finalize_solstice_dcf file to its case-specific asset-manifest digest and reject evidence copied from another task, even when its filename and business vocabulary look plausible. Enforced by globally unique asset SHA-256 census and manifest membership.
21
investigation · investigation_structured_exportsReconcile the fa_finalize_solstice_dcf structured exports (dcf_assumptions-system-export.csv, dcf_models-system-export.csv, fa_companies-system-export.csv, financial_periods-system-export.csv, lbo_models-system-export.csv, review_notes-system-export.csv) with the live records they represent; no single export is sufficient when related tables disagree. Enforced by exact table selectors, live-state snapshot, and parser-validated CSV records.
22
investigation · investigation_controlled_workbookUse the parser-validated review-calendar workbook for fa_finalize_solstice_dcf to test timing and capacity assumptions against current records, while treating workbook conclusions as evidence rather than authorization. Enforced by XLSX parser gate, workbook digest, and exact temporal selectors.
23
correlation · correlation_communication_authority_splitFor fa_finalize_solstice_dcf, distinguish what the source request, operations Slack thread, and former-owner message say from what the signed approval and current control actually authorize. Enforced by communication-record correlation plus current-authority and approval-scope checks.
24
correlation · correlation_provider_live_lineageJoin the task-scoped Gmail, Drive, Sheets, and Slack records for fa_finalize_solstice_dcf to the exact live-domain selectors by case id, provider id, immutable row id, and effective revision; a same-name file or row from another task is not equivalent evidence. Enforced by provider asset ids and SHA-256 manifest joined to argument-aware domain selectors.
25
alternatives · alternatives_blast_radiusCompare the complete table-level blast radius of the supported fa_finalize_solstice_dcf branch with the hold and broad-cleanup alternatives; reject any option that changes an unapproved neighbor even if its headline answer is correct. Enforced by all-table before/after digests and unauthorized-write control.
26
decision · decision_citation_lineageCarry the selected fa_finalize_solstice_dcf decision into the result row with the exact current evidence IDs used to derive it, so a correct-looking answer backed by stale or cross-task evidence fails. Enforced by exact result evidence set and wrong-evidence control.
27
alternatives · alternatives_supportedKeep the bounded evidence-supported outcome for fa_finalize_solstice_dcf available only if effective authority and current records agree. Enforced by exact final-state and result contract.
28
alternatives · alternatives_holdRetain a hold/escalate branch for fa_finalize_solstice_dcf if controlling records conflict or authority is absent; difficulty alone is not a reason to hold. Enforced by wrong-decision negative control.
29
alternatives · alternatives_broadReject a broad neighboring-record cleanup for fa_finalize_solstice_dcf, even when similar names or stale notes make it superficially plausible. Enforced by unauthorized-write negative control.
30
alternatives · alternatives_no_changeEvaluate a no-domain-change outcome for fa_finalize_solstice_dcf: retain it only when the exact live predicates do not authorize a mutation; otherwise it is an incomplete state-only or shortcut branch. Enforced by task-specific source VCode plus no-op and state-only controls.
31
decision · decision_supported_branchDerive the selected fa_finalize_solstice_dcf branch from the intersection of authority, identity, relationship, timing, and live-state evidence—not from the prompt's requested outcome. Enforced by source VCode, exact state digests, and task-result row.
32
decision · decision_task_specific_causal_pathThe deterministic reasoning for fa_finalize_solstice_dcf is task-specific: Finalise the Solstice base-case DCF (model_id 9201, company 9001). Total the free_cash_flow across every financial period belonging to company 9001 — other companies' periods do not count. Set model 9201's enterprise_value to that total and its status to 'final', mark its WACC assumption (assumption_id 9301) as status 'verified', and file the review: create a review note on model 9201 from reviewer 'Head of Research' with status 'open' and a body containing the summed cash flow. The model must establish each predicate from the provider evidence and live domain records, then carry only the supported branch into state. Enforced by exact source VCode, authored selectors, before/after state, and adversarial branch controls.
33
procedure · procedure_successful_evidenceEvery exact contextual selector required for fa_finalize_solstice_dcf must succeed; a failed or differently filtered call is not equivalent evidence. Enforced by argument-aware successful trace.
34
procedure · procedure_provider_discovery_before_detailFor fa_finalize_solstice_dcf, complete the task-scoped Gmail, Drive, and Slack discovery calls before relying on exact message, file, spreadsheet, or thread records; the verifier accepts equivalent causal ordering but requires every exact discovery and detail selector before state changes. Enforced by provider list/search and exact-get trace selectors before first mutation.
35
procedure · procedure_read_before_writeComplete all 52 causal reads for fa_finalize_solstice_dcf before the first domain mutation or persisted completion result. Enforced by trace sequence bounds.
36
procedure · procedure_result_after_workPersist the exact fa_finalize_solstice_dcf result only after any supported domain transition has completed. Enforced by source-mutation-before-submit sequence check.
37
procedure · procedure_post_write_readbackReopen the exact persisted fa_finalize_solstice_dcf result after submission so verification observes stored state rather than trusting the write response. Enforced by exact get_task_result selector after submit.
38
state · state_dcf_assumptionsProduce only the supported fa_finalize_solstice_dcf transition in dcf_assumptions: added assumption_id=none; removed assumption_id=none; updated ["assumption_id=9301 (status:'unverified'→'verified')"]. The complete final table must match digest a281fbc8f92f0729558ab2e6d87079c04df66b27bcca5535f19e284677601eca. Enforced by exact complete-table digest and source VCode.
39
state · state_dcf_modelsProduce only the supported fa_finalize_solstice_dcf transition in dcf_models: added model_id=none; removed model_id=none; updated ["model_id=9201 (enterprise_value:0.0→360.0, status:'draft'→'final')"]. The complete final table must match digest 4f59701265ff9d6d201de4451cec88a49fd3071f4f9345fbdf7d5b10b3dfdea0. Enforced by exact complete-table digest and source VCode.
40
state · state_review_notesProduce only the supported fa_finalize_solstice_dcf transition in review_notes: added note_id=['9502']; removed note_id=none; updated none. The complete final table must match digest c50be13852569399bb762c8cbd9d7d16dbb10fe80adad004056a00af26618354. Enforced by exact complete-table digest and source VCode.
41
state · state_authorized_transition_01The fa_finalize_solstice_dcf state transition with exact arguments {"enterprise_value": 360.0, "model_id": 9201, "status": "final"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
42
state · state_authorized_transition_02The fa_finalize_solstice_dcf state transition with exact arguments {"assumption_id": 9301, "status": "verified"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
43
state · state_authorized_transition_03The fa_finalize_solstice_dcf state transition with exact arguments {"body": "Unlevered free cash flow across the period set totals 360.", "model_id": 9201, "reviewer_name": "Head of Research", "status": "open"} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
44
state · state_authorized_transition_04The fa_finalize_solstice_dcf state transition with exact arguments {"channel": "#research-review", "client_msg_id": "081722302081d2d5b6262be43f41918d", "text": "Finalize model 9201 at enterprise value 360, verify WACC assumption 9301, and open the Head of Research review note. I reconciled 98 candidate rows across 5 live record families, excluded 92 neighbor rows, and retained 6 immutable task records. The selected outcome is scheduled for 2026-09-03 (-1 days versus the 2026-09-04 need date), costs $2600, fits with 4 internal hours to spare, and is authorized by APR-dd2b7f5d040bc9e4. The broader neighbor remediation requires additional approval and was not executed. Exact answer: 360."} must succeed and be reflected in the final audited tables. Enforced by successful exact MCP trace plus complete-table digest.
45
state · state_source_outcomeAll 2 task-specific executable outcome assertions for fa_finalize_solstice_dcf must hold against the final sandbox state. Enforced by checked-in deterministic VCode.
46
state · state_result_rowCreate exactly one task-result row for fa_finalize_solstice_dcf containing the supported decision, exact summary, evidence IDs, causal workpaper, and exact answer when applicable. Enforced by persisted task_results state.
47
answer · answer_result_submittedReturn an audit-ready fa_finalize_solstice_dcf conclusion in the persisted result row rather than only narrating intended work. Enforced by exact task-result state verifier.
48
answer · answer_evidence_citationsCite the current policy, source request, controlled workbook, and effective source document for fa_finalize_solstice_dcf; stale authority cannot substitute. Enforced by exact required evidence set.
49
answer · answer_exact_valueReturn the exact task-supported scalar for fa_finalize_solstice_dcf, derived from the live records rather than copied from an asset. Enforced by deterministic submitted-answer equality.
50
containment · containment_write_scopeUse only the fa_finalize_solstice_dcf authorized domain mutation set and the task-result handoff; no neighboring table or record may change. Enforced by allowed-write trace plus all-table digests.
51
containment · containment_no_rejected_mutationNo state-changing request in fa_finalize_solstice_dcf may fail and then be hidden by a later correct-looking final answer. Enforced by successful mutation trace.
52
identity · identity_H01_scope_identityLocate fa_finalize_solstice_dcf through case DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF and the exact immutable target map {"dcf_assumptions": [9301], "dcf_models": [9201], "fa_companies": [9001], "financial_periods": [9101, 9102, 9103], "review_notes": []}. Enforced by exact workpaper.scope plus successful argument-aware search/count/get evidence.
53
decision · decision_H02_requirement_derivationApply the controlled task rule from operating-policy.md#task-specific-decision-control and derive all 5 required record families before choosing an outcome. Enforced by exact workpaper.requirement and D02.
54
investigation · investigation_H03_availability_reconciliationReconcile 98 observed candidate rows, 92 excluded neighbors, and 6 usable targets. Enforced by exact workpaper.reconciliation and D03-D05.
55
decision · decision_H04_gap_or_boundNet 5 required families against 5 covered families into exact gap 0. Enforced by exact workpaper.gapOrBound and D06.
56
authority · authority_H05_external_constraintApply independent approval APR-dd2b7f5d040bc9e4 and its $2800 cost boundary. Enforced by exact workpaper.externalConstraint corroborated by approval email and Drive record.
57
investigation · investigation_H06_internal_constraintApply the 2026-09-04 business need, 17 available hours, and $200 hourly rate from the controlled calendar. Enforced by exact workpaper.internalConstraint and D07-D10.
58
alternatives · alternatives_H07_qualified_optionsGrade all three task-specific alternatives with exact outcome, date, signed variance, incremental cost, authority status, and one selected branch. Enforced by exact workpaper.options equality and wrong-reasoning control.
59
decision · decision_H08_recommendationSelect evidence-backed-scoped-outcome with exact outcome 'Finalize model 9201 at enterprise value 360, verify WACC assumption 9301, and open the Head of Research review note.'. Enforced by exact workpaper.recommendation and D13.
60
decision · decision_H09_control_comparisonCompare outcome 2026-09-03 to need 2026-09-04 into signed variance -1 and status ON_OR_BEFORE_NEED. Enforced by independently recomputed exact workpaper.controlComparison and D11.
61
authority · authority_H10_authority_applicationApply authority APR-dd2b7f5d040bc9e4 to the selected immutable scope and reject broad-neighbor-remediation. Enforced by exact workpaper.authorityApplication and D12.
62
state · state_H11_controlled_executionExecute exactly 3 authorized domain writes across 3 changed tables, perform 3 exact-record provider readbacks, then reopen the persisted result. Enforced by exact writes, complete-table digests, provider-native exact-record readbacks, persisted result, and result readback trace.
63
answer · answer_H12_stakeholder_communicationPost the exact stakeholder handoff to #research-review carrying the selected outcome, timing, cost, capacity constraint, and authority. Enforced by exact slack.chat_postMessage call and persisted stakeholder-message row.
64
answer · answer_H13_exact_intermediatesPersist all 13 dependent derivations and every intermediate requirement, reconciliation, constraint, option, authority, state, and answer field exactly. Enforced by exact causal-workpaper equality; wrong-reasoning negative control.
65
containment · containment_visible_hidden_partitionThe 28 files visible for fa_finalize_solstice_dcf contain evidence only; verifier digests, the oracle trajectory, and precomputed outcomes remain outside the asset room. Enforced by release-tree partition and leakage gate.

Generator label only; no empirical model difficulty claim is published.

Episode trace

Follow the tools, outputs, and verdict.

Switch between a successful reference replay and a real failing floor episode. Both start from the same immutable world state; neither is a model leaderboard row.

deterministic solvability ceiling

Reference solver

A checked-in v3 causal investigation replayed against a fresh SQLite copy. This proves the task can pass; it is not a model submission.

Task: fa_finalize_solstice_dcf
Steps: 61
Reward: 1.0
Status: passed

00
Task loadeduser → agent
Fresh seeded database copied into an isolated episode.

gmail.messages.listtool success

Arguments

{
  "maxResults": 100,
  "q": "\"DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF\"",
  "userId": "me"
}

Result

{
  "messages": [
    {
      "id": "msg-6a02c41ec2c7908f56f9",
      "threadId": "msg-6a02c41ec2c7908f56f9"
    },
    {
      "id": "msg-a3859fdd8b6dde1dfd91",
      "threadId": "msg-a3859fdd8b6dde1dfd91"
    },
    {
      "id": "msg-369888fa4cd226854a04",
      "threadId": "msg-369888fa4cd226854a04"
    }
  ],
  "resultSizeEstimate": 3
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-6a02c41ec2c7908f56f9",
  "userId": "me"
}

Result

{
  "id": "msg-6a02c41ec2c7908f56f9",
  "name": "source-request.eml",
  "mimeType": "message/rfc822",
  "sha256": "dcd4775007ce0c1d5b158c01a28e6ad04de30242181f907b19ab8862963a1479",
  "bytes": 813
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-a3859fdd8b6dde1dfd91",
  "userId": "me"
}

Result

{
  "id": "msg-a3859fdd8b6dde1dfd91",
  "name": "former-owner-suggestion.eml",
  "mimeType": "message/rfc822",
  "sha256": "83a7a3d6c45695af93b663b53d3997806b8750aa0f83858e9b73c074352af3b3",
  "bytes": 635
}

gmail.messages.gettool success

Arguments

{
  "format": "raw",
  "id": "msg-369888fa4cd226854a04",
  "userId": "me"
}

Result

{
  "id": "msg-369888fa4cd226854a04",
  "name": "approval-scope.eml",
  "mimeType": "message/rfc822",
  "sha256": "167a45ac06457fda01bfcf045f3f141bb592094dd3c2ad0bf8ab7d9c5c8eddf8",
  "bytes": 835
}

google_drive.files.listtool success

Arguments

{
  "fields": "files(id,name,mimeType,modifiedTime,version)",
  "pageSize": 1000,
  "q": "fullText contains 'DB24-FILINGBENCH-50-FA_FINALIZE_SOLSTICE_DCF'"
}

Result

{
  "files": [
    {
      "id": "file-4f7f602567f25f0ceced",
      "name": "operating-policy.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-85daaaa1f655f828efdb",
      "name": "issuer-filing-extract.pdf",
      "mimeType": "application/pdf",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "sheet-6754468a021f983acee3",
      "name": "valuation-review-model.xlsx",
      "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-ba1befb3f34aacd98f87",
      "name": "approval-decision.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-449aa4d3f2962992baca",
      "name": "drive-source-index.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-bc744fb4e743886f16a7",
      "name": "provider-contracts.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-c17b25ec14d4ce70c9de",
      "name": "drive-approval-record.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-76d1aff002da28751d67",
      "name": "authority-revision-2026-3.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-2a857ed446abda210217",
      "name": "authority-revision-2025-4.md",
      "mimeType": "text/markdown",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-a8861e3d491909302593",
      "name": "control-standard-2025-4.pdf",
      "mimeType": "application/pdf",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "sheet-1d5a4ee07edbe874c9be",
      "name": "review-calendar.xlsx",
      "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-f4e565b8ccc3b9f169d6",
      "name": "record-lineage.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-4c07dac7f1d5542c90a4",
      "name": "system-audit.log",
      "mimeType": "text/plain",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-58f10bcdad11474ca4c3",
      "name": "live-snapshot.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-070d9a3082911d004d8c",
      "name": "evidence-status.yaml",
      "mimeType": "application/yaml",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-64a8b9e4e6f86561a91e",
      "name": "custodian-notes.txt",
      "mimeType": "text/plain",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-0a1551c275585b2dc5ed",
      "name": "fa_companies-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-be74ae297db504cbda48",
      "name": "dcf_models-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-f725914ea935e75b3d25",
      "name": "financial_periods-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-685a07277edb13d22e96",
      "name": "dcf_assumptions-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-c307c47e0a4aca47b561",
      "name": "review_notes-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-ae994e57f782cf6f7a89",
      "name": "lbo_models-system-export.csv",
      "mimeType": "text/csv",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    },
    {
      "id": "file-58f8fe6ef69d7e523545",
      "name": "asset-manifest.json",
      "mimeType": "application/json",
      "modifiedTime": "2026-08-29T09:00:00Z",
      "version": "1"
    }
  ]
}