Executable checksFail
Recorded benchmark evaluator · devopsbench-100:3.2.7:original-verifier
| correctness.every_match_transitioned: satisfied | Met |
| correctness.nothing_extra_transitioned: satisfied | Met |
| correctness.outcome_recorded: satisfied | Met |
| deployment.read_the_source_of_truth: satisfied | Met |
| quality.ticket_closed: close ticket ENG-2806 once the work is done | Not met |
| quality.closed_after_the_work: close ENG-2806 only once the work is done, not before | Not met |
| correctness.scoped_change_no_fabricated_data: satisfied | Met |
| correctness.world_invariants_intact: satisfied | Met |
| quality.no_new_hardcoded_secrets: satisfied | Met |
| quality.no_unproductive_ci_loops: satisfied | Met |
| correctness.v4_case_identity_resolved: resolve DOB-036 through its immutable Jira key, linked GitHub issue, and cross-tracker relationship | Not met |
| correctness.v4_authority_reconciled: compare current control OPS-CONTROL-2026.03 with retired control OPS-CONTROL-2025.11 before acting | Not met |
| correctness.v4_live_state_correlated: correlate the task-specific api-gateway provider state instead of relying on files or tracker text alone | Not met |
| correctness.v4_material_evidence_complete: complete all 20 materially causal evidence joins inside the larger reference investigation before changing state | Not met |
| deployment.v4_evidence_before_state_change: establish every material fact before the first successful task state transition | Not met |
| correctness.v4_supported_path_selected: derive and execute the task-supported branch rather than the stale-note or broad-workaround alternatives | Not met |
| correctness.v4_source_mutations_completed: complete each task-required state-changing capability at least once successfully | Not met |
| deployment.v4_state_readbacks_complete: reopen the persisted provider and tracker state after the final source mutation instead of trusting acknowledgements | Not met |
| correctness.v4_completion_handoff_exactly_once: write exactly one scoped completion handoff for DOB-036 | Not met |
| deployment.v4_source_work_precedes_readback: perform provider readback only after the task-specific operational state change | Not met |
| deployment.v4_readback_precedes_handoff: post the completion handoff only after the changed provider state has been reopened | Not met |
| deployment.v4_handoff_read_back: reopen the exact case-room conversation after posting the completion handoff | Not met |
| deployment.v4_no_rejected_mutations: complete without a rejected state-changing call; failed exploratory reads remain recoverable | Not met |
| correctness.v5_capacity_plan_recorded: satisfied | Met |
| correctness.v5_answer_business_need_date: satisfied | Met |
| correctness.v5_answer_replicas_per_zone: satisfied | Met |
| correctness.v5_answer_production_zones: satisfied | Met |
| correctness.v5_answer_required_replicas: satisfied | Met |
| correctness.v5_answer_observed_replicas: satisfied | Met |
| correctness.v5_answer_reserved_replicas: satisfied | Met |
| correctness.v5_answer_usable_replicas: satisfied | Met |
| correctness.v5_answer_replica_gap: satisfied | Met |
| correctness.v5_answer_quantity_unit: satisfied | Met |
| correctness.v5_answer_standard_capacity_date: satisfied | Met |
| correctness.v5_answer_expedited_capacity_date: satisfied | Met |
| correctness.v5_answer_capacity_request_replicas: satisfied | Met |
| correctness.v5_answer_next_change_window: satisfied | Met |
| correctness.v5_answer_standard_plan_completion: satisfied | Met |
| correctness.v5_answer_expedited_plan_completion: satisfied | Met |
| correctness.v5_answer_reserved_release_completion: Calculated release_reserved_capacity outcome as 2026-03-03 and kept its separate-approval condition. | Not met |
| correctness.v5_answer_recommended_option: Compared the date, cost and authority of standard_capacity_plan, expedite_capacity and release_reserved_capacity; selected expedite_capacity as the best currently authorised plan under the readiness standard. | Not met |
| correctness.v5_answer_recommended_outcome_date: satisfied | Met |
| correctness.v5_answer_recommended_incremental_cost_usd: satisfied | Met |
| correctness.v5_answer_escalation_approval_required: satisfied | Met |
| correctness.v5_answer_approval_reference: satisfied | Met |
| correctness.v5_answer_outcome_vs_control_days: satisfied | Met |
| correctness.v5_answer_decision_timing_status: satisfied | Met |
| correctness.v5_capacity_evidence_complete: read the readiness standard, CloudCap order VEND-36, change approval CHG-36, the Linear reservation and the status-page cutover notice before changing state | Not met |
| deployment.v5_decision_evidence_precedes_record: derive the capacity plan only after every source fact it depends on has been read successfully | Not met |
| deployment.v5_record_precedes_handoff: satisfied | Met |
| correctness.v5_approval_applied_to_selected_scope: apply CHG-36 to expedite_capacity only and keep release_reserved_capacity outside current authority | Not met |
| correctness.v5_handoff_is_natural_completion: satisfied | Met |
| correctness.v5_handoff_states_selected_option: state the selected option (expedite_capacity) in the scoped completion handoff | Not met |
| correctness.v5_handoff_states_outcome_date: state the outcome date (2026-03-04) in the scoped completion handoff | Not met |
| correctness.v5_handoff_states_approval_reference: satisfied | Met |
| correctness.v5_handoff_states_timing_status: state the timing status (ON_TIME) in the scoped completion handoff | Not met |
| correctness.v5_handoff_states_binding_constraint: state the binding constraint (CloudCap expedited capacity on 2026-03-04) in the scoped completion handoff | Not met |
passed=0.0; pass threshold=1.0 investigation.scope; investigation.authority; investigation.live_state; analysis.causal_reasoning; analysis.capacity_plan; decision.supported_path; decision.options; state.primary; state.coordination; verification.outcome; verification.readback; execution.sequence; answer.insights; execution.efficiency; execution.delivery Partial native reward: 0.3636. The recorded native verdict determines task success.