Skip to benchmark
Blobfish AI
BenchmarksTalk to usOpen Studio
Blobfish ResearchERPBench-100 v1.0.0Public release

Can an agent run the ERP transaction end to end—not just read the record?

ERPBench-100 measures transactional ERP execution across ten synthetic tenants running an Oracle-Fusion-shaped system of record. The agent must find the operative batch, note, roster or policy, reconcile it against live ERP records, recompute the quantities and amounts, decide what policy actually permits, commit the sales order, receipt, requisition, receipt request, invoice, document, absence, hire or price change, log the register, and leave a review-only handoff.

Inspect qualification and leaderboard Browse all 100 tasks
Run on Harbor ↗Dataset on Hugging Face ↗Executable source ↗
100ERP workflows
10synthetic ERP tenants
52provider-shaped tools
460released source assets
0LLM grading calls
Release gate passed

The tasks are solvable; the qualification controls are not model rows. All 100 oracle episodes reached 100 ERPScore, all 100/100 deterministic replays matched, 600 adversarial episodes across 6 control families produced 0 strict false accepts, and no untouched tenant seed satisfied any ERP-state criterion. A ranked model appears only after a complete version-pinned run on this exact release.

Assets per task
28
Evidence reads
15
Exact criteria
34
Native formats
8
Grounded workflows, clean-room data

The ten workflow families mirror the ERP archetypes that real customers run through Nario's dataflywheel—order import, shipment verification, unpaid-invoice reconciliation, reorder monitoring, procure-to-pay, document expiry, work-report rollups, channel-order sync, recruitment quotas, and effective-dated record supersession. Only the shape of that work is reused: every tenant, person, item, quantity, amount, document and message is synthetic. Public Enterprise-Bench, ERP-Bench, APEX and Archipelago materials informed what a reproducible ERP benchmark should expose; nothing gated was downloaded or scraped.

Enterprise-Bench L1-L2 ↗ERP-Bench (Odoo) ↗APEX Agents ↗APEX Accounting ↗Archipelago ↗

Measured performance

Leaderboard

ERPScore is the mean deterministic outcome score across all 100 tasks. Qualification controls stay below the ranked table so an oracle can never masquerade as a model result.

Ranking

Model leaderboard

Only complete, version-pinned model runs on this exact release receive a rank.

RankAgentCoverageScoreRun
No complete version-pinned model run is published for ERPBench-100 yet. The executed controls below establish a 100-point ceiling and six distinct failure floors without receiving ranks.
Qualification evidence

Measured controls

Oracle and adversarial replays prove solvability and verifier discrimination. They are measured, but never ranked as model submissions.

ClassControlCoverageScoreEvidence
REF
Reference oracleDeterministic release qualification
100 tasks
100.0%
28.4 calls avg.Solvability ceiling; not a model submission.
CTRL
No-op controlDeterministic release qualification
100 tasks
4.0%
calls avg.Executed adversarial control; never ranked with models.
CTRL
Answer-only shortcutDeterministic release qualification
100 tasks
46.0%
calls avg.Executed adversarial control; never ranked with models.
CTRL
State-only shortcutDeterministic release qualification
100 tasks
85.0%
calls avg.Executed adversarial control; never ranked with models.
CTRL
Wrong-source controlDeterministic release qualification
100 tasks
88.0%
calls avg.Executed adversarial control; never ranked with models.
CTRL
Wrong-target controlDeterministic release qualification
100 tasks
82.8%
calls avg.Executed adversarial control; never ranked with models.
CTRL
Process-as-received controlDeterministic release qualification
100 tasks
89.5%
calls avg.Executed adversarial control; never ranked with models.
15%Discovery
25%ERP calculation
15%Decision
20%Committed ERP state
10%Register and handoff
10%Readback
5%Containment

100 distinct tasks

Task browser

Every task is an independently authored, high-level employee request over one frozen ERP tenant. Open any row to inspect the prompt, 28-file evidence room, policy alternatives, and all 34 checks.

100 shown
erpbench-001Customer order import

Import the customer's consolidated order

**Northbridge Fastener Works · Toledo plant · Customer order import** Cascade Assembly Inc. sent their consolidated CA-PO-77310 order as a spreadsheet this morning and wants it confirmed today. Get it into the system properly: check every line against what we already imported for that purchase order and against the items we still sell, price it from the current list rather than whatever the spreadsheet says, and respect the customer's credit standing before you submit anything. Log the import in the ops register and get a reply to Dana Whitfield ready for review. Use the evidence available in the connected systems. Leave unrelated records and other tenants unchanged, keep outbound communication in review status, and do not claim completion until the durable state and readbacks agree.

Company
NORTHBRIDGE
As of
2026-02-09T09:00:00Z
Seeded files
28
Reference solution
27 MCP calls

Complete task prompt

34 graded criteria summarized
**Northbridge Fastener Works · Toledo plant · Customer order import**

Cascade Assembly Inc. sent their consolidated CA-PO-77310 order as a spreadsheet this morning and wants it confirmed today. Get it into the system properly: check every line against what we already imported for that purchase order and against the items we still sell, price it from the current list rather than whatever the spreadsheet says, and respect the customer's credit standing before you submit anything. Log the import in the ops register and get a reply to Dana Whitfield ready for review.

Use the evidence available in the connected systems. Leave unrelated records and other tenants unchanged, keep outbound communication in review status, and do not claim completion until the durable state and readbacks agree.
How the employee outcome is evaluated

Reasoning, persisted state, and the answer must agree.

7 semantic milestones

An operative-source and ERP-calculation chain ending in committed Oracle-shaped transaction state, a register row, a decision record, and review-only communication.

Strict success
The exact current sources, recomputed quantities and amounts, decision, committed ERP records, register and handoff, readbacks, and containment all agree.
Ordering policy
Exact call order is not graded; required investigations must precede the first controlled ERP write and each write must be read back.
Inspect the task-specific causal milestones
  1. discovery
    discovery

    Discovery contributes 15 ERPScore points.

  2. calculation
    calculation

    ERP calculation contributes 25 ERPScore points.

  3. decision
    decision

    Decision contributes 15 ERPScore points.

  4. erp_state
    erp_state

    Committed ERP state contributes 20 ERPScore points.

  5. handoff
    handoff

    Register and handoff contributes 10 ERPScore points.

  6. readback
    readback

    Readback contributes 10 ERPScore points.

  7. containment
    containment

    Containment contributes 5 ERPScore points.

Decision space (4 grounded options)
  • Import the accepted lines at current list prices and submit the order — selected: Matches the operative source, the recomputed quantities and the tenant's policy.
  • Import the accepted lines but hold submission for credit release: Policy-shaped alternative whose conditions the current evidence does not support.
  • Import every spreadsheet line at the spreadsheet prices: Ignores duplicates, tolerance, expiry, quota, band or supersession rules and writes unsupported state.
  • Stop and escalate the whole batch without importing: Fails to distinguish the controlled work the analyst is authorized to complete from the items that need approval.
Inspect all 34 deterministic criteria
  • discovery: Before any controlled write, read the task contract (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the operating request in the mailbox (any listed route counts). (1 pts)
  • discovery: Before any controlled write, establish which sources are current authority (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the ops channel thread for the task (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the ops register before writing to it (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the item master or current price list (any listed route counts). (1 pts)
  • discovery: Before any controlled write, index the shared drive for current and superseded versions (any listed route counts). (1 pts)
  • discovery: Before any controlled write, locate the consolidated order batch (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the consolidated order batch (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the earlier partial import (any listed route counts). (1 pts)
  • discovery: Before any controlled write, check the sales orders already on the purchase order (any listed route counts). (1 pts)
  • discovery: Before any controlled write, check the customer's credit standing (any listed route counts). (1 pts)
  • discovery: Before any controlled write, recognise the superseded price list (any listed route counts). (1 pts)
  • discovery: Before any controlled write, read the audit trail for the purchase order (any listed route counts). (1 pts)
  • discovery: Before any controlled write, confirm the customer master record (any listed route counts). (1 pts)
  • calculation: Submit the exact evidence-grounded accepted lines (5). (5 pts)
  • calculation: Submit the exact evidence-grounded rejected lines (3). (5 pts)
  • calculation: Submit the exact evidence-grounded imported units (3160). (5 pts)
  • calculation: Submit the exact evidence-grounded order total (2757.6). (5 pts)
  • calculation: Submit the exact evidence-grounded duplicate lines (2). (5 pts)
  • decision: Select the policy-supported option import_accepted_lines_and_submit. (10 pts)
  • decision: Ground the decision in the operative source FILE-NBR-ORDER-BATCH-0209 rather than a superseded version. (5 pts)
  • erp_state: Persist the primary ERP transaction with the exact provider-critical values. (10 pts)
  • erp_state: Persist the workflow-specific secondary ERP state (submission, hold, validation, alerts, or absence records) and nothing that policy forbids. (4 pts)
  • erp_state: Write the exact task row to the controlled ops register range. (3 pts)
  • erp_state: Record the decision with status, primary record, rationale and source references. (3 pts)
  • handoff: Save a review-only email draft that states the option, the key figures and the primary record. (5 pts)
  • handoff: Post the review handoff to the ops channel with the decision status. (5 pts)
  • readback: Read the ERP record back after the primary write. (3 pts)
  • readback: Read back the register row and the decision record after committing them. (3 pts)
  • readback: Read back the email draft and channel history before submission. (2 pts)
  • readback: Read back the durable task submission. (2 pts)
  • containment: Leave every unrelated tenant record unchanged and make no rejected calls. (3 pts)
  • containment: Keep outbound communication in review status and never apply an unauthorized credit, hire or price change. (2 pts)
Open released task dataset

Seeded evidence

Asset room

Order batches, prior partial imports, current and superseded price lists, pick confirmations, remittance advices, aging, policies, delivery notes, supplier invoices, compliance checklists, rosters, work reports, channel exports, headcount approvals, candidate registers, price batches, memos and audit logs are all visible.

01-consolidated-order-batch.xlsxassets/northbridgeOperativeXLSX4.5 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
02-prior-partial-import.xlsxassets/northbridgeCorroboratingXLSX2.9 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
03-current-price-list.csvassets/northbridgeOperativeCSV404 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
04-superseded-price-list.csvassets/northbridgeCorroboratingCSV443 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
05-pick-confirmation.csvassets/northbridgeOperativeCSV421 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
06-remittance-advice.pdfassets/northbridgeOperativePDF950 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
07-ar-aging.xlsxassets/northbridgeCorroboratingXLSX3.4 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
08-credit-policy.mdassets/northbridgeCorroboratingMD447 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
09-reorder-policy.xlsxassets/northbridgeOperativeXLSX3.6 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
10-demand-forecast.csvassets/northbridgeCorroboratingCSV221 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
11-supplier-invoice.pdfassets/northbridgeCorroboratingPDF982 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
12-delivery-note.pdfassets/northbridgeOperativePDF1.0 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
13-ap-match-policy.mdassets/northbridgeCorroboratingMD378 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
14-compliance-checklist.xlsxassets/northbridgeOperativeXLSX5.2 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
15-document-alert-log.csvassets/northbridgeCorroboratingCSV138 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
16-shift-roster.xlsxassets/northbridgeOperativeXLSX4.8 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
17-shift-work-reports.csvassets/northbridgeCorroboratingCSV296 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
18-channel-export.jsonassets/northbridgeOperativeJSON1.2 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
19-customer-capture-chat.jsonassets/northbridgeCorroboratingJSON417 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
20-headcount-approval.pdfassets/northbridgeOperativePDF963 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
21-candidate-register.xlsxassets/northbridgeCorroboratingXLSX4.0 KB
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
22-wage-table.csvassets/northbridgeCorroboratingCSV91 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
23-price-batch.csvassets/northbridgeOperativeCSV463 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
24-authority-memo.emlassets/northbridgeCorroboratingEML532 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
25-source-map.yamlassets/northbridgeCorroboratingYAML470 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
26-ops-audit.logassets/northbridgeCorroboratingLOG290 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
task-brief.mdassets/tasks/erpbench-001OperativeMD938 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file
starting-snapshot.jsonassets/tasks/erpbench-001CorroboratingJSON400 B
Agent-visible synthetic evidence; current and superseded records are deliberately mixed.Open released file

Runnable world

Environment and tool contract

Six MCP servers expose one isolated SQLite tenant: Oracle-Fusion-shaped order management, shipping, receivables, inventory, procurement, payables and HCM resources next to Gmail, Drive, Sheets and Slack. Every controlled write is durable, task-scoped, and checked against the complete before/after state.

MCP package pinerpbench100@1.0.0
MCP protocol2025-06-18
Catalog SHA-25675cefba8f5010f30a17043e871e0e1561515262ea7440e16d1d471303b8fca04
Harbor package SHA-256c24d160e25113c1cb53635886c4a07f0a8204b52c992c8c1683ba17e3035071f
Hugging Face commit0bafbb17f9de10f08c8fa9ffd79ac5fc18d253ef
Synthetic worldfusion-erp-tenants-v1
erpbench.get_task
Read onlyIdempotentClosed sandbox

Read the task-scoped outcome contract without revealing the gold answer.

Documented provider operationMCP erpbenchOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "task_id": {
      "description": "Task identifier",
      "type": "string"
    }
  },
  "required": [
    "task_id"
  ],
  "type": "object"
}
erpbench.get_decision
Read onlyIdempotentClosed sandbox

Read back the durable decision record for a task.

Documented provider operationMCP erpbenchOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "task_id": {
      "description": "Task identifier",
      "type": "string"
    }
  },
  "required": [
    "task_id"
  ],
  "type": "object"
}
erpbench.get_submission
Read onlyIdempotentClosed sandbox

Read back the durable submitted answer.

Documented provider operationMCP erpbenchOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "task_id": {
      "description": "Task identifier",
      "type": "string"
    }
  },
  "required": [
    "task_id"
  ],
  "type": "object"
}
erpbench.record_decision
State changingNon-idempotentClosed sandbox

Record the selected option, status, primary ERP record, rationale and source references.

Documented provider operationMCP erpbenchOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "decision": {
      "description": "Selected option",
      "type": "string"
    },
    "primary_record": {
      "description": "Primary ERP record number",
      "type": "string"
    },
    "rationale": {
      "description": "Rationale",
      "type": "string"
    },
    "source_refs": {
      "description": "Source references",
      "items": {
        "type": "string"
      },
      "type": "array"
    },
    "status": {
      "description": "Decision status",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "decision",
    "status",
    "primary_record",
    "rationale",
    "source_refs"
  ],
  "type": "object"
}
erpbench.submit_answer
State changingNon-idempotentClosed sandbox

Persist the structured final answer for deterministic grading.

Documented provider operationMCP erpbenchOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "answers": {
      "additionalProperties": true,
      "description": "Task-specific answer",
      "type": "object"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "answers"
  ],
  "type": "object"
}
oracle_fusion.items.list
Read onlyIdempotentClosed sandbox

Get items for an inventory organization.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/itemsV2Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.items.update
State changingNon-idempotentClosed sandbox

Update the list price of one item as of an effective date.

Documented provider operationPATCH /fscmRestApi/resources/11.13.18.05/itemsV2/{itemsV2UniqID}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "EffectiveDate": {
      "description": "ISO effective date",
      "type": "string"
    },
    "ItemId": {
      "description": "Item identifier",
      "type": "integer"
    },
    "ListPrice": {
      "description": "New list price",
      "type": "number"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "ItemId",
    "ListPrice",
    "EffectiveDate"
  ],
  "type": "object"
}
oracle_fusion.customer_account_activities.get
Read onlyIdempotentClosed sandbox

Get one customer account with open receivables and credit status.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/receivablesCustomerAccountActivities/{AccountId}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "AccountId": {
      "description": "Customer account number",
      "type": "string"
    }
  },
  "required": [
    "AccountId"
  ],
  "type": "object"
}
oracle_fusion.sales_orders.list
Read onlyIdempotentClosed sandbox

Get sales orders from Order Management.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/salesOrdersForOrderHubOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.sales_orders.get
Read onlyIdempotentClosed sandbox

Get one sales order with its lines.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/salesOrdersForOrderHub/{OrderKey}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "OrderKey": {
      "description": "Order number",
      "type": "string"
    }
  },
  "required": [
    "OrderKey"
  ],
  "type": "object"
}
oracle_fusion.sales_orders.create
State changingNon-idempotentClosed sandbox

Create one sales order; SubmittedFlag false leaves it as a draft awaiting release.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/salesOrdersForOrderHubOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "BuyingPartyName": {
      "description": "Customer name when no account exists",
      "type": "string"
    },
    "BuyingPartyNumber": {
      "description": "Customer account number",
      "type": "string"
    },
    "CustomerPONumber": {
      "description": "Customer purchase order",
      "type": "string"
    },
    "OrderTotal": {
      "description": "Header total when lines are not itemized",
      "type": "number"
    },
    "RequestedFulfillmentOrganizationCode": {
      "description": "Fulfillment organization",
      "type": "string"
    },
    "SourceTransactionNumber": {
      "description": "Source document reference",
      "type": "string"
    },
    "SourceTransactionSystem": {
      "description": "Source system",
      "type": "string"
    },
    "SubmittedFlag": {
      "description": "Submit the order",
      "type": "boolean"
    },
    "TransactionType": {
      "description": "Order type",
      "type": "string"
    },
    "lines": {
      "description": "Order lines: ProductNumber, OrderedQuantity, OrderedUOM, UnitListPrice",
      "items": {},
      "type": "array"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "SourceTransactionNumber",
    "SourceTransactionSystem",
    "CustomerPONumber",
    "TransactionType",
    "RequestedFulfillmentOrganizationCode",
    "SubmittedFlag"
  ],
  "type": "object"
}
oracle_fusion.shipments.list
Read onlyIdempotentClosed sandbox

Get shipments.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/shipmentsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.shipment_lines.list
Read onlyIdempotentClosed sandbox

Get shipment lines with requested and shipped quantities.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/shipmentLinesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.shipment_lines.update
State changingNon-idempotentClosed sandbox

Update the shipped quantity of one unconfirmed shipment line.

Documented provider operationPATCH /fscmRestApi/resources/11.13.18.05/shipmentLines/{ShipmentLine}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "Comments": {
      "description": "Reason",
      "type": "string"
    },
    "ShipmentLine": {
      "description": "Shipment line identifier",
      "type": "integer"
    },
    "ShippedQuantity": {
      "description": "Corrected shipped quantity",
      "type": "integer"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "ShipmentLine",
    "ShippedQuantity"
  ],
  "type": "object"
}
oracle_fusion.receivables_invoices.list
Read onlyIdempotentClosed sandbox

Get receivables invoices.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/receivablesInvoicesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.receivables_invoices.get
Read onlyIdempotentClosed sandbox

Get one receivables invoice.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/receivablesInvoices/{CustomerTransactionId}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "CustomerTransactionId": {
      "description": "Transaction identifier",
      "type": "integer"
    }
  },
  "required": [
    "CustomerTransactionId"
  ],
  "type": "object"
}
oracle_fusion.standard_receipts.list
Read onlyIdempotentClosed sandbox

Get standard receipts for a customer account.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/standardReceiptsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.standard_receipts.create
State changingNon-idempotentClosed sandbox

Create a standard receipt and apply it through remittance references.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/standardReceiptsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "BusinessUnit": {
      "description": "Business unit",
      "type": "string"
    },
    "Currency": {
      "description": "Currency",
      "type": "string"
    },
    "CustomerAccountNumber": {
      "description": "Customer account number",
      "type": "string"
    },
    "ReceiptAmount": {
      "description": "Receipt amount",
      "type": "number"
    },
    "ReceiptDate": {
      "description": "ISO receipt date",
      "type": "string"
    },
    "ReceiptMethod": {
      "description": "Receipt method",
      "type": "string"
    },
    "ReceiptNumber": {
      "description": "Receipt number",
      "type": "string"
    },
    "remittanceReferences": {
      "description": "Rows of ReferenceType, ReferenceNumber, ApplyAmount",
      "items": {},
      "type": "array"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "ReceiptNumber",
    "ReceiptAmount",
    "ReceiptDate",
    "CustomerAccountNumber",
    "ReceiptMethod",
    "BusinessUnit",
    "Currency",
    "remittanceReferences"
  ],
  "type": "object"
}
oracle_fusion.onhand_balances.list
Read onlyIdempotentClosed sandbox

Get on-hand and reserved quantities.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/inventoryOnhandBalancesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.purchase_orders.list
Read onlyIdempotentClosed sandbox

Get purchase orders.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/purchaseOrdersOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.purchase_orders.get
Read onlyIdempotentClosed sandbox

Get one purchase order.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/purchaseOrders/{purchaseOrdersUniqID}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "purchaseOrdersUniqID": {
      "description": "Purchase order number",
      "type": "string"
    }
  },
  "required": [
    "purchaseOrdersUniqID"
  ],
  "type": "object"
}
oracle_fusion.purchase_order_lines.list
Read onlyIdempotentClosed sandbox

Get the lines of one purchase order.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/purchaseOrders/{purchaseOrdersUniqID}/child/linesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "purchaseOrdersUniqID": {
      "description": "Purchase order number",
      "type": "string"
    }
  },
  "required": [
    "purchaseOrdersUniqID"
  ],
  "type": "object"
}
oracle_fusion.suppliers.list
Read onlyIdempotentClosed sandbox

Get suppliers with lead times.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/suppliersOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.purchase_requisitions.create
State changingNon-idempotentClosed sandbox

Create a purchase requisition with lines.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/purchaseRequisitionsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "Description": {
      "description": "Description",
      "type": "string"
    },
    "Justification": {
      "description": "Justification",
      "type": "string"
    },
    "Preparer": {
      "description": "Preparer",
      "type": "string"
    },
    "RequisitioningBU": {
      "description": "Requisitioning business unit",
      "type": "string"
    },
    "lines": {
      "description": "Lines: ItemNumber, Quantity, UOM, UnitPrice, Supplier, RequestedDeliveryDate",
      "items": {},
      "type": "array"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "RequisitioningBU",
    "Preparer",
    "Description",
    "Justification",
    "lines"
  ],
  "type": "object"
}
oracle_fusion.purchase_requisitions.submit
State changingNon-idempotentClosed sandbox

Submit a requisition for approval.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/purchaseRequisitions/{purchaseRequisitionsUniqID}/action/submitRequisitionOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "purchaseRequisitionsUniqID": {
      "description": "Requisition number",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "purchaseRequisitionsUniqID"
  ],
  "type": "object"
}
oracle_fusion.purchase_requisitions.get
Read onlyIdempotentClosed sandbox

Get one purchase requisition.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/purchaseRequisitions/{purchaseRequisitionsUniqID}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "purchaseRequisitionsUniqID": {
      "description": "Requisition number",
      "type": "string"
    }
  },
  "required": [
    "purchaseRequisitionsUniqID"
  ],
  "type": "object"
}
oracle_fusion.receiving_receipt_requests.list
Read onlyIdempotentClosed sandbox

Get receiving receipt requests.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/receivingReceiptRequestsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.receiving_receipt_requests.create
State changingNon-idempotentClosed sandbox

Create a receiving receipt request for delivered purchase-order lines.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/receivingReceiptRequestsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "OrganizationCode": {
      "description": "Organization",
      "type": "string"
    },
    "ReceiptSourceCode": {
      "description": "Receipt source",
      "type": "string"
    },
    "VendorName": {
      "description": "Supplier",
      "type": "string"
    },
    "lines": {
      "description": "Lines: DocumentNumber, DocumentLineNumber, ItemNumber, Quantity, TransactionType",
      "items": {},
      "type": "array"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "ReceiptSourceCode",
    "OrganizationCode",
    "lines"
  ],
  "type": "object"
}
oracle_fusion.invoices.list
Read onlyIdempotentClosed sandbox

Get Payables invoices.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/invoicesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.invoices.get
Read onlyIdempotentClosed sandbox

Get one Payables invoice with holds.

Documented provider operationGET /fscmRestApi/resources/11.13.18.05/invoices/{invoicesUniqID}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "invoicesUniqID": {
      "description": "Invoice number",
      "type": "string"
    }
  },
  "required": [
    "invoicesUniqID"
  ],
  "type": "object"
}
oracle_fusion.invoices.create
State changingNon-idempotentClosed sandbox

Create one Payables invoice matched to purchase-order lines.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/invoicesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "BusinessUnit": {
      "description": "Business unit",
      "type": "string"
    },
    "InvoiceAmount": {
      "description": "Invoice amount",
      "type": "number"
    },
    "InvoiceCurrency": {
      "description": "Currency",
      "type": "string"
    },
    "InvoiceDate": {
      "description": "ISO invoice date",
      "type": "string"
    },
    "InvoiceNumber": {
      "description": "Invoice number",
      "type": "string"
    },
    "Supplier": {
      "description": "Supplier",
      "type": "string"
    },
    "invoiceLines": {
      "description": "Lines: LineNumber, PurchaseOrderNumber, PurchaseOrderLineNumber, ItemNumber, Quantity, UnitPrice",
      "items": {},
      "type": "array"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "BusinessUnit",
    "Supplier",
    "InvoiceNumber",
    "InvoiceAmount",
    "InvoiceCurrency",
    "InvoiceDate",
    "invoiceLines"
  ],
  "type": "object"
}
oracle_fusion.invoices.validate
State changingNon-idempotentClosed sandbox

Validate an invoice for payment.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/invoices/action/validateInvoiceOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "BusinessUnit": {
      "description": "Business unit",
      "type": "string"
    },
    "InvoiceNumber": {
      "description": "Invoice number",
      "type": "string"
    },
    "ProcessAction": {
      "description": "Must be Validate",
      "type": "string"
    },
    "Supplier": {
      "description": "Supplier",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "ProcessAction",
    "BusinessUnit",
    "Supplier",
    "InvoiceNumber"
  ],
  "type": "object"
}
oracle_fusion.invoice_holds.create
State changingNon-idempotentClosed sandbox

Place one Payables invoice on hold with a reason.

Documented provider operationPOST /fscmRestApi/resources/11.13.18.05/invoiceHoldsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "HoldName": {
      "description": "Hold name",
      "type": "string"
    },
    "HoldReason": {
      "description": "Hold reason",
      "type": "string"
    },
    "InvoiceId": {
      "description": "Invoice identifier",
      "type": "integer"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "InvoiceId",
    "HoldName",
    "HoldReason"
  ],
  "type": "object"
}
oracle_fusion.workers.list
Read onlyIdempotentClosed sandbox

Get workers for a legal employer.

Documented provider operationGET /hcmRestApi/resources/11.13.18.05/workersOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.workers.create
State changingNon-idempotentClosed sandbox

Hire one worker under a standard contract.

Documented provider operationPOST /hcmRestApi/resources/11.13.18.05/workersOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "CandidateId": {
      "description": "Candidate identifier",
      "type": "string"
    },
    "ContractEndDate": {
      "description": "ISO contract end",
      "type": "string"
    },
    "DisplayName": {
      "description": "Display name",
      "type": "string"
    },
    "HireDate": {
      "description": "ISO hire date",
      "type": "string"
    },
    "JobCode": {
      "description": "Job",
      "type": "string"
    },
    "LegalEmployerName": {
      "description": "Legal employer",
      "type": "string"
    },
    "MonthlySalary": {
      "description": "Monthly salary",
      "type": "number"
    },
    "PersonNumber": {
      "description": "Person number",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "PersonNumber",
    "DisplayName",
    "LegalEmployerName",
    "JobCode",
    "HireDate",
    "ContractEndDate",
    "MonthlySalary",
    "CandidateId"
  ],
  "type": "object"
}
oracle_fusion.document_records.list
Read onlyIdempotentClosed sandbox

Get worker documents of record with expiry and check status.

Documented provider operationGET /hcmRestApi/resources/11.13.18.05/documentRecordsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.document_records.update
State changingNon-idempotentClosed sandbox

Update the status or verification flag of one document record.

Documented provider operationPATCH /hcmRestApi/resources/11.13.18.05/documentRecords/{DocumentRecordId}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "DocumentRecordId": {
      "description": "Document record identifier",
      "type": "integer"
    },
    "Status": {
      "description": "New status, e.g. ALERTED",
      "type": "string"
    },
    "VerifiedFlag": {
      "description": "Mark the mandatory check verified",
      "type": "boolean"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "DocumentRecordId"
  ],
  "type": "object"
}
oracle_fusion.absences.list
Read onlyIdempotentClosed sandbox

Get absence records.

Documented provider operationGET /hcmRestApi/resources/11.13.18.05/absencesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Provider query expression, e.g. Field='value' and Other='value'",
      "type": "string"
    }
  },
  "required": [],
  "type": "object"
}
oracle_fusion.absences.create
State changingNon-idempotentClosed sandbox

Record one absence for a worker.

Documented provider operationPOST /hcmRestApi/resources/11.13.18.05/absencesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "absenceStatusCd": {
      "description": "Status code",
      "type": "string"
    },
    "absenceType": {
      "description": "Absence type",
      "type": "string"
    },
    "employer": {
      "description": "Legal employer",
      "type": "string"
    },
    "endDate": {
      "description": "ISO end",
      "type": "string"
    },
    "personNumber": {
      "description": "Person number",
      "type": "string"
    },
    "startDate": {
      "description": "ISO start",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "personNumber",
    "absenceType",
    "startDate",
    "endDate",
    "employer",
    "absenceStatusCd"
  ],
  "type": "object"
}
gmail.messages.list
Read onlyIdempotentClosed sandbox

Search mailbox messages.

Documented provider operationGET /gmail/v1/users/{userId}/messagesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Gmail search query",
      "type": "string"
    }
  },
  "required": [
    "q"
  ],
  "type": "object"
}
gmail.messages.get
Read onlyIdempotentClosed sandbox

Read one mailbox message.

Documented provider operationGET /gmail/v1/users/{userId}/messages/{id}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "id": {
      "description": "Message identifier",
      "type": "string"
    }
  },
  "required": [
    "id"
  ],
  "type": "object"
}
gmail.drafts.create
State changingNon-idempotentClosed sandbox

Save a review-only email draft; drafts are never sent by the sandbox.

Documented provider operationPOST /gmail/v1/users/{userId}/draftsOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "body": {
      "description": "Body",
      "type": "string"
    },
    "subject": {
      "description": "Subject",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    },
    "to": {
      "description": "Recipient",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "to",
    "subject",
    "body"
  ],
  "type": "object"
}
gmail.drafts.get
Read onlyIdempotentClosed sandbox

Read back one draft.

Documented provider operationGET /gmail/v1/users/{userId}/drafts/{id}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "id": {
      "description": "Draft identifier",
      "type": "string"
    }
  },
  "required": [
    "id"
  ],
  "type": "object"
}
google_drive.files.list
Read onlyIdempotentClosed sandbox

List files; supports name contains 'text'.

Documented provider operationGET /drive/v3/filesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "q": {
      "description": "Drive query",
      "type": "string"
    }
  },
  "required": [
    "q"
  ],
  "type": "object"
}
google_drive.files.get
Read onlyIdempotentClosed sandbox

Read file metadata, version and current-authority flag.

Documented provider operationGET /drive/v3/files/{fileId}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "fileId": {
      "description": "File identifier",
      "type": "string"
    }
  },
  "required": [
    "fileId"
  ],
  "type": "object"
}
google_drive.files.download
Read onlyIdempotentClosed sandbox

Download the content of one file.

Documented provider operationGET /drive/v3/files/{fileId}?alt=mediaOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "fileId": {
      "description": "File identifier",
      "type": "string"
    }
  },
  "required": [
    "fileId"
  ],
  "type": "object"
}
google_sheets.spreadsheets.values.get
Read onlyIdempotentClosed sandbox

Read a range from the ops workbook.

Documented provider operationGET /v4/spreadsheets/{spreadsheetId}/values/{range}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "range": {
      "description": "A1 range",
      "type": "string"
    },
    "spreadsheetId": {
      "description": "Spreadsheet identifier",
      "type": "string"
    }
  },
  "required": [
    "spreadsheetId",
    "range"
  ],
  "type": "object"
}
google_sheets.spreadsheets.values.update
State changingNon-idempotentClosed sandbox

Write one controlled range; other cells are preserved.

Documented provider operationPUT /v4/spreadsheets/{spreadsheetId}/values/{range}Open provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "range": {
      "description": "A1 range",
      "type": "string"
    },
    "spreadsheetId": {
      "description": "Spreadsheet identifier",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    },
    "valueInputOption": {
      "description": "RAW or USER_ENTERED",
      "type": "string"
    },
    "values": {
      "description": "Two-dimensional values",
      "items": {},
      "type": "array"
    }
  },
  "required": [
    "task_id",
    "spreadsheetId",
    "range",
    "valueInputOption",
    "values"
  ],
  "type": "object"
}
google_sheets.spreadsheets.values.append
State changingNon-idempotentClosed sandbox

Append rows to a tab.

Documented provider operationPOST /v4/spreadsheets/{spreadsheetId}/values/{range}:appendOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "range": {
      "description": "Tab range",
      "type": "string"
    },
    "spreadsheetId": {
      "description": "Spreadsheet identifier",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    },
    "valueInputOption": {
      "description": "RAW or USER_ENTERED",
      "type": "string"
    },
    "values": {
      "description": "Rows to append",
      "items": {},
      "type": "array"
    }
  },
  "required": [
    "task_id",
    "spreadsheetId",
    "range",
    "valueInputOption",
    "values"
  ],
  "type": "object"
}
slack.conversations_history
Read onlyIdempotentClosed sandbox

Read a channel's messages, including task posts.

Documented provider operationPOST /api/conversations.historyOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "channel": {
      "description": "Channel",
      "type": "string"
    },
    "task_id": {
      "description": "Filter to one task's posts",
      "type": "string"
    }
  },
  "required": [
    "channel"
  ],
  "type": "object"
}
slack.search_messages
Read onlyIdempotentClosed sandbox

Search messages across channels.

Documented provider operationPOST /api/search.messagesOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "query": {
      "description": "Search query",
      "type": "string"
    }
  },
  "required": [
    "query"
  ],
  "type": "object"
}
slack.chat_postMessage
State changingNon-idempotentClosed sandbox

Post the review handoff to a channel.

Documented provider operationPOST /api/chat.postMessageOpen provider documentation 
{
  "additionalProperties": false,
  "properties": {
    "channel": {
      "description": "Channel",
      "type": "string"
    },
    "review_status": {
      "description": "Must remain draft_for_review",
      "type": "string"
    },
    "task_id": {
      "description": "Active task identifier; every write must carry it",
      "type": "string"
    },
    "text": {
      "description": "Message",
      "type": "string"
    }
  },
  "required": [
    "task_id",
    "channel",
    "text",
    "review_status"
  ],
  "type": "object"
}

Architecture comparison

Public ERP benchmark anchors, deterministic ERPBench implementation

The public architecture is preserved where it improves reproducibility; the tenant corpus, tool surface and grading implementation are independent.

Inspect Archipelago
LayerPublic ERP/APEX benchmark patternERPBench-100
WorldEnterprise ERP scenario with seeded records and documentsTen frozen synthetic Oracle-Fusion-shaped tenants with 28 task-visible files
EnvironmentContainer, MCP gateway, seeded application stateHarbor task, six MCP servers, task-local SQLite tenant snapshot
TrajectoryMessages, tool calls, record edits, final stateFull provider-shaped calls, outputs, before/after state and verdict
GradingEnterprise-Bench's LLM judge; ERP-Bench's undocumented verification34 executable calculation, source, state, readback and containment checks in a deterministic executable verifier; no judge model
ReleaseTask packages, seeded worlds, task metadataHF mirror, Harbor dataset, gold contracts, digests and ten public oracle traces

Replay the work

Trajectories

Ten full reference trajectories show the request, provider-shaped calls, results, ERP writes, register and handoff, readbacks, and the deterministic verdict. They prove execution, not model capability.

AgentReference oracleDeterministic release solver
Score100.0%Strict pass
Tool calls27Provider-native MCP trace
Run cost—
29 events
Full transcript Verifier report Source trial artifact
  1. Employee request

    **Northbridge Fastener Works · Toledo plant · Customer order import** Cascade Assembly Inc. sent their consolidated CA-PO-77310 order as a spreadsheet this morning and wants it confirmed today. Get it into the system properly: check every line against what we already imported for that purchase order and against the items we still sell, price it from the current list rather than whatever the spreadsheet says, and respect the customer's credit standing before you submit anything. Log the import in the ops register and get a reply to Dana Whitfield ready for review. Use the evidence available in the connected systems. Leave unrelated records and other tenants unchanged, keep outbound communication in review status, and do not claim completion until the durable state and readbacks agree.

  2. 1
    Scopeerpbench.get_taskok
    {
  "task_id": "erpbench-001"
}

    {"allowed_write_tools": ["oracle_fusion.sales_orders.create", "oracle_fusion.shipment_lines.update", "oracle_fusion.standard_receipts.create", "oracle_fusion.purchase_requisitions.create", "oracle_fusion.purchase_requisitions.submit", "oracle_fusion.receiving_receipt_requests.create", "oracle_fusion.invoices.create", "oracle_fusion.invoices.validate", "oracle_fusion.invoice_holds.create", "oracle_fusion.document_records.update", "oracle_fusion.absences.create", "oracle_fusion.workers.create", "o…

  3. 2
    Investigategmail.messages.getok
    {
  "id": "MSG-erpbench-001-REQUEST"
}

    {"body": "**Northbridge Fastener Works · Toledo plant · Customer order import**\n\nCascade Assembly Inc. sent their consolidated CA-PO-77310 order as a spreadsheet this morning and wants it confirmed today. Get it into the system properly: check every line against what we already imported for that purchase order and against the items we still sell, price it from the current list rather than whatever the spreadsheet says, and respect the customer's credit standing before you submit anything. Log …

  4. 3
    Investigategmail.messages.getok
    {
  "id": "MSG-NBR-AUTHORITY-0209"
}

    {"body": "From: marcus.lee@northbridge-sim.example\nTo: analyst@northbridge-sim.example\nSubject: Northbridge Fastener Works operating authority for February\n\nCurrent sources of truth: the 2026-02-09 order batch, the 2026-02-08 pick confirmation, the 2026-02-06 remittance advice, reorder policy R4, the 2026-02-06 delivery note, the 2026-02-09 compliance checklist, the 2026-02-08 roster, the 2026-02-09 channel export, the 2026 headcount approval and the Q2 price batch. Earlier versions are supe…

  5. 4
    Investigateslack.conversations_historyok
    {
  "channel": "#ops-nbr"
}

    {"channel": "#ops-nbr", "messages": [{"text": "Partial import for CA-PO-77310 went in on 2026-02-03 as SO-NBR-40039; the rest of the lines are still pending.", "thread_ts": "THREAD-NBR-IMPORT", "ts": "2026-02-03T10:20:00Z", "user": "Marcus Lee"}, {"text": "Please resolve erpbench-001 against the current February authority; keep the handoff in review.", "thread_ts": "THREAD-erpbench-001", "ts": "2026-02-09T09:00:00Z", "user": "Marcus Lee"}, {"text": "Operative sources are listed in the source map…

  6. 5
    Investigategoogle_sheets.spreadsheets.values.getok
    {
  "range": "Register!A1:H40",
  "spreadsheetId": "SHEET-NBR-OPS"
}

    {"range": "Register!A1:H40", "spreadsheetId": "SHEET-NBR-OPS", "values": [["Record", "Option", "Values", "", "", "", "", "Source"], ["SO-NBR-DECOY", "closed_manual_order", 1, 0, 1280.0, 0, 0, "MANUAL-2026-01"]]}

  7. 6
    Investigateoracle_fusion.items.listok
    {
  "q": "OrganizationCode='NBR1'"
}

    {"count": 6, "items": [{"ItemDescription": "12 x 100 wedge anchor", "ItemId": 300031, "ItemNumber": "ANC-12-100", "ItemStatusValue": "Active", "ListPrice": 2.45, "OrganizationCode": "NBR1", "PriceEffectiveDate": "2026-01-01", "PrimaryUOMValue": "EA"}, {"ItemDescription": "M10 x 60 hex bolt, zinc", "ItemId": 300010, "ItemNumber": "FST-M10-60", "ItemStatusValue": "Active", "ListPrice": 1.32, "OrganizationCode": "NBR1", "PriceEffectiveDate": "2026-01-01", "PrimaryUOMValue": "EA"}, {"ItemDescription…

  8. 7
    Investigategoogle_drive.files.listok
    {
  "q": "name contains 'policy'"
}

    {"files": [{"id": "FILE-NBR-CREDIT-POLICY-R6", "isCurrentAuthority": true, "mimeType": "text/markdown", "modifiedTime": "2026-02-09T06:00:00Z", "name": "08-credit-policy.md", "version": 2}, {"id": "FILE-NBR-REORDER-POLICY-R4", "isCurrentAuthority": true, "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "modifiedTime": "2026-02-09T06:00:00Z", "name": "09-reorder-policy.xlsx", "version": 2}, {"id": "FILE-NBR-AP-MATCH-POLICY", "isCurrentAuthority": true, "mimeType": …

  9. 8
    Investigategoogle_drive.files.listok
    {
  "q": "name contains 'CA-PO-77310'"
}

    {"files": [{"id": "FILE-NBR-ORDER-BATCH-0209", "isCurrentAuthority": true, "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "modifiedTime": "2026-02-09T06:00:00Z", "name": "01-consolidated-order-batch.xlsx", "version": 2}, {"id": "FILE-NBR-ORDER-BATCH-0203", "isCurrentAuthority": false, "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "modifiedTime": "2026-02-03T10:00:00Z", "name": "02-prior-partial-import.xlsx", "version": 2}, {"id…

  10. 9
    Investigategoogle_drive.files.downloadok
    {
  "fileId": "FILE-NBR-ORDER-BATCH-0209"
}

    {"content": "Line,Customer PO,Item,Description,Quantity,Unit price (customer sheet)\r\n1,CA-PO-77310,FST-M8-40,\"M8 x 40 hex bolt, zinc\",1200,0.89\r\n2,CA-PO-77310,FST-M10-60,\"M10 x 60 hex bolt, zinc\",800,1.4\r\n3,CA-PO-77310,NUT-M8-NYL,M8 nylon lock nut,640,0.2\r\n4,CA-PO-77310,WSH-M10-SS,\"M10 flat washer, stainless\",300,0.12\r\n5,CA-PO-77310,FST-M8-40,\"M8 x 40 hex bolt, zinc\",450,0.89\r\n6,CA-PO-77310,ANC-12-100,12 x 100 wedge anchor,220,2.6\r\n7,CA-PO-77310,FST-M6-20-OLD,M6 x 20 hex bo…

  11. 10
    Investigategoogle_drive.files.downloadok
    {
  "fileId": "FILE-NBR-ORDER-BATCH-0203"
}

    {"content": "Line,Customer PO,Item,Quantity,Imported order,Imported on\r\n5,CA-PO-77310,FST-M8-40,450,SO-NBR-40039,2026-02-03\r\n8,CA-PO-77310,NUT-M8-NYL,500,SO-NBR-40039,2026-02-03\r\n", "id": "FILE-NBR-ORDER-BATCH-0203", "isCurrentAuthority": false, "mimeType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "name": "02-prior-partial-import.xlsx"}

  12. 11
    Investigateoracle_fusion.sales_orders.listok
    {
  "q": "CustomerPONumber='CA-PO-77310'"
}

    {"count": 1, "items": [{"BuyingPartyName": "Cascade Assembly Inc.", "BuyingPartyNumber": "CUST-10412", "CreatedBy": "ops_lead", "CreationDate": "2026-02-03T10:12:00Z", "CustomerPONumber": "CA-PO-77310", "LastUpdateDate": "2026-02-03T10:12:00Z", "OrderKey": "SO-NBR-40039", "OrderNumber": "SO-NBR-40039", "OrderTotal": 473.0, "RequestedFulfillmentOrganizationCode": "NBR1", "SourceTransactionNumber": "FILE-NBR-ORDER-BATCH-0203", "SourceTransactionSystem": "OPS", "StatusCode": "Awaiting Shipping", "S…

  13. 12
    Investigateoracle_fusion.customer_account_activities.getok
    {
  "AccountId": "CUST-10412"
}

    {"AccountId": "CUST-10412", "AsOfDate": "2026-02-09", "CreditHoldFlag": false, "CreditLimit": 180000.0, "CustomerName": "Cascade Assembly Inc.", "OpenReceivablesAmount": 73050.0, "OpenTransactions": 5, "PastDueAmount": 50830.0, "PaymentTerms": "Net 30"}

  14. 13
    Investigategoogle_drive.files.downloadok
    {
  "fileId": "FILE-NBR-PRICE-LIST-2025H2"
}

    {"content": "item,description,list_price,status,effective_from,superseded\nFST-M8-40,M8 x 40 hex bolt, zinc,0.80,Active,2025-07-01,true\nFST-M10-60,M10 x 60 hex bolt, zinc,1.25,Active,2025-07-01,true\nNUT-M8-NYL,M8 nylon lock nut,0.18,Active,2025-07-01,true\nWSH-M10-SS,M10 flat washer, stainless,0.10,Active,2025-07-01,true\nANC-12-100,12 x 100 wedge anchor,2.33,Active,2025-07-01,true\nFST-M6-20-OLD,M6 x 20 hex bolt (discontinued),0.49,Active,2025-07-01,true\n", "id": "FILE-NBR-PRICE-LIST-2025H2"…

  15. 14
    Investigategoogle_drive.files.downloadok
    {
  "fileId": "FILE-NBR-OPS-AUDIT-LOG"
}

    {"content": "2026-02-03T10:12:00Z NBR partial import created SO-NBR-40039 from an earlier copy of CA-PO-77310\n2026-02-06T15:40:00Z NBR delivery note DN-NBR-0206 received at Toledo plant\n2026-02-08T22:05:00Z NBR pick confirmation posted for SHP-NBR-4471\n2026-02-09T06:00:00Z NBR Shopify export generated\n", "id": "FILE-NBR-OPS-AUDIT-LOG", "isCurrentAuthority": true, "mimeType": "text/plain", "name": "26-ops-audit.log"}

  16. 15
    Investigategoogle_sheets.spreadsheets.values.getok
    {
  "range": "Customers!A1:E12",
  "spreadsheetId": "SHEET-NBR-OPS"
}

    {"range": "Customers!A1:E12", "spreadsheetId": "SHEET-NBR-OPS", "values": [["Customer", "Number", "Email", "Tax id", "Channel"], ["Cascade Assembly Inc.", "CUST-10412", "ap@cascade-sim.example", "TX-NBR-0001", "direct"], ["Industrial Outlet Co.", "CUST-NBR-DECOY1", "buyer@outlet-sim.example", "TX-NBR-0002", "direct"]]}

  17. 16
    Executeoracle_fusion.sales_orders.createok
    {
  "BuyingPartyNumber": "CUST-10412",
  "CustomerPONumber": "CA-PO-77310",
  "RequestedFulfillmentOrganizationCode": "NBR1",
  "SourceTransactionNumber": "FILE-NBR-ORDER-BATCH-0209",
  "SourceTransactionSystem": "OPS",
  "SubmittedFlag": true,
  "TransactionType": "Standard Orders",
  "lines": [
    {
      "OrderedQuantity": 1200,
      "OrderedUOM": "EA",
      "ProductNumber": "FST-M8-40",
      "UnitListPrice": 0.84
    },
    {
      "OrderedQuantity": 800,
      "OrderedUOM": "EA",
      "ProductNumber": "FST-M10-60",
      "UnitListPrice": 1.32
    },
    {
      "OrderedQuantity": 640,
      "OrderedUOM": "EA",
      "ProductNumber": "NUT-M8-NYL",
      "UnitListPrice": 0.19
    },
    {
      "OrderedQuantity": 300,
      "OrderedUOM": "EA",
      "ProductNumber": "WSH-M10-SS",
      "UnitListPrice": 0.11
    },
    {
      "OrderedQuantity": 220,
      "OrderedUOM": "EA",
      "ProductNumber": "ANC-12-100",
      "UnitListPrice": 2.45
    }
  ],
  "task_id": "erpbench-001"
}

    {"BuyingPartyName": "Cascade Assembly Inc.", "BuyingPartyNumber": "CUST-10412", "CreatedBy": "analyst", "CreationDate": "2026-02-09T09:00:00Z", "CustomerPONumber": "CA-PO-77310", "LastUpdateDate": "2026-02-09T09:00:00Z", "OrderKey": "SO-NBR-50011", "OrderNumber": "SO-NBR-50011", "OrderTotal": 2757.6, "RequestedFulfillmentOrganizationCode": "NBR1", "SourceTransactionNumber": "FILE-NBR-ORDER-BATCH-0209", "SourceTransactionSystem": "OPS", "StatusCode": "Booked", "SubmittedFlag": true, "TransactionT…

  18. 17
    Investigateoracle_fusion.sales_orders.getok
    {
  "OrderKey": "SO-NBR-50011"
}

    {"BuyingPartyName": "Cascade Assembly Inc.", "BuyingPartyNumber": "CUST-10412", "CreatedBy": "analyst", "CreationDate": "2026-02-09T09:00:00Z", "CustomerPONumber": "CA-PO-77310", "LastUpdateDate": "2026-02-09T09:00:00Z", "OrderKey": "SO-NBR-50011", "OrderNumber": "SO-NBR-50011", "OrderTotal": 2757.6, "RequestedFulfillmentOrganizationCode": "NBR1", "SourceTransactionNumber": "FILE-NBR-ORDER-BATCH-0209", "SourceTransactionSystem": "OPS", "StatusCode": "Booked", "SubmittedFlag": true, "TransactionT…

  19. 18
    Handoffgoogle_sheets.spreadsheets.values.updateok
    {
  "range": "Register!A21:H21",
  "spreadsheetId": "SHEET-NBR-OPS",
  "task_id": "erpbench-001",
  "valueInputOption": "RAW",
  "values": [
    [
      "SO-NBR-50011",
      "import_accepted_lines_and_submit",
      5,
      3,
      3160,
      2757.6,
      2,
      "FILE-NBR-ORDER-BATCH-0209"
    ]
  ]
}

    {"changeId": "CHANGE-erpbench-001-1", "spreadsheetId": "SHEET-NBR-OPS", "updatedRange": "Register!A21:H21", "updatedRows": 1}

  20. 19
    Decideerpbench.record_decisionok
    {
  "decision": "import_accepted_lines_and_submit",
  "primary_record": "SO-NBR-50011",
  "rationale": "Current evidence supports import_accepted_lines_and_submit; operative source FILE-NBR-ORDER-BATCH-0209.",
  "source_refs": [
    "FILE-NBR-ORDER-BATCH-0209",
    "MSG-erpbench-001-REQUEST",
    "MSG-NBR-AUTHORITY-0209"
  ],
  "status": "submitted",
  "task_id": "erpbench-001"
}

    {"decision": "import_accepted_lines_and_submit", "decision_id": "DECISION-erpbench-001", "primary_record": "SO-NBR-50011", "rationale": "Current evidence supports import_accepted_lines_and_submit; operative source FILE-NBR-ORDER-BATCH-0209.", "source_refs": ["FILE-NBR-ORDER-BATCH-0209", "MSG-erpbench-001-REQUEST", "MSG-NBR-AUTHORITY-0209"], "status": "submitted", "task_id": "erpbench-001"}

  21. 20
    Handoffgmail.drafts.createok
    {
  "body": "Northbridge Fastener Works: recommend import_accepted_lines_and_submit (submitted). accepted lines 5; rejected lines 3; imported units 3160; order total 2757.6; duplicate lines 2. Primary record SO-NBR-50011; operative source FILE-NBR-ORDER-BATCH-0209. Draft for review, not sent.",
  "subject": "erpbench-001 — Import the customer's consolidated order — for review",
  "task_id": "erpbench-001",
  "to": "marcus.lee@northbridge-sim.example"
}

    {"id": "DRAFT-erpbench-001", "message": {"labelIds": ["DRAFT"], "subject": "erpbench-001 — Import the customer's consolidated order — for review", "to": "marcus.lee@northbridge-sim.example"}, "review_status": "draft_for_review"}

  22. 21
    Handoffslack.chat_postMessageok
    {
  "channel": "#ops-nbr",
  "review_status": "draft_for_review",
  "task_id": "erpbench-001",
  "text": "erpbench-001: submitted / import_accepted_lines_and_submit / SO-NBR-50011 — review draft ready."
}

    {"channel": "#ops-nbr", "message": {"review_status": "draft_for_review", "text": "erpbench-001: submitted / import_accepted_lines_and_submit / SO-NBR-50011 — review draft ready."}, "ok": true, "ts": "2026-02-09T09:00:00Z"}

  23. 22
    Investigategoogle_sheets.spreadsheets.values.getok
    {
  "range": "Register!A21:H21",
  "spreadsheetId": "SHEET-NBR-OPS"
}

    {"range": "Register!A21:H21", "spreadsheetId": "SHEET-NBR-OPS", "values": [["SO-NBR-50011", "import_accepted_lines_and_submit", 5, 3, 3160, 2757.6, 2, "FILE-NBR-ORDER-BATCH-0209"]]}

  24. 23
    Verifyerpbench.get_decisionok
    {
  "task_id": "erpbench-001"
}

    {"decision": "import_accepted_lines_and_submit", "decision_id": "DECISION-erpbench-001", "primary_record": "SO-NBR-50011", "rationale": "Current evidence supports import_accepted_lines_and_submit; operative source FILE-NBR-ORDER-BATCH-0209.", "source_refs": ["FILE-NBR-ORDER-BATCH-0209", "MSG-erpbench-001-REQUEST", "MSG-NBR-AUTHORITY-0209"], "status": "submitted", "task_id": "erpbench-001"}

Methodology

How the numbers are made

01

Ten deep ERP tenants

Each synthetic tenant carries an Oracle-Fusion-shaped item master, customers and suppliers, sales orders, a staged shipment, open receivables, on-hand balances and open purchase orders, a supplier invoice, worker documents, a shift roster, a channel export, a candidate register and a price batch, plus current and superseded Drive files, a mailbox, an ops register workbook and chat channels. Ten workflows reuse each frozen tenant as an operations team would.

02

High-level employee requests

Prompts ask for a business outcome and a review-ready handoff. They do not prescribe tool order or leak record identifiers. The agent must find the operative source among superseded versions, recompute quantities and amounts from current evidence, and decide what policy allows it to commit.

03

Stateful cross-application execution

Six logical MCP servers expose Oracle-Fusion-shaped order management, shipping, receivables, inventory, procurement, payables and HCM resources plus Gmail, Google Drive, Google Sheets, Slack and benchmark controls over isolated SQLite state. Order, shipment, receipt, requisition, receiving, invoice, hold, document, absence, worker, price, register, decision and communication writes are durable and task-scoped.

04

One deterministic ERPScore

ERPScore allocates 100 executable points to discovery, ERP calculation, decision quality, committed ERP state, register and handoff, readback and containment. No LLM judge or exact reference call sequence is used.

05

Two-sided qualification

The release executed 800 episodes: 100/100 oracle runs, 100/100 exact deterministic replays, and 600 adversarial controls across six families (no-op, answer-only, state-only, wrong-source, wrong-target, process-as-received) with 0 false accepts. A pre-satisfied-seed gate confirms that no untouched tenant already satisfies a calculation, decision or ERP-state criterion.

06

Nario dataflywheel grounding and clean-room boundary

Workflow archetypes, tool shapes and data shapes were observed through Nario's production dataflywheel traces; no customer content, name, identifier, message or value was reused. Public Enterprise-Bench, ERP-Bench, APEX and Archipelago pages informed the release contract; Mercor's gated dataset was not downloaded or scraped. Every tenant, prompt, file, value, tool, answer and trajectory in ERPBench-100 is newly synthetic at production shape.

07

Leaderboard honesty

Qualification controls prove solvability and discrimination but are never ranked as models. A model row appears only after a complete version-pinned 100-task run has an inspectable receipt.

08

Model runs on this release

Disclosed pilot (not ranked): GPT-5.6 Luna (Codex 0.151.0, max reasoning) on 10 of 100 tasks scored a mean 85.00 ERPScore (3 strict passes); the full 100-task run is required before a leaderboard row appears.

Qualification controls — excluded from the leaderboard
  • Reference oracle: 100.0% across 100 tasks — Solvability ceiling; not a model submission.
  • No-op control: 4.0% across 100 tasks — Executed adversarial control; never ranked with models.
  • Answer-only shortcut: 46.0% across 100 tasks — Executed adversarial control; never ranked with models.
  • State-only shortcut: 85.0% across 100 tasks — Executed adversarial control; never ranked with models.
  • Wrong-source control: 88.0% across 100 tasks — Executed adversarial control; never ranked with models.
  • Wrong-target control: 82.8% across 100 tasks — Executed adversarial control; never ranked with models.
  • Process-as-received control: 89.5% across 100 tasks — Executed adversarial control; never ranked with models.

Executable ERP benchmark

Production-shaped ERP workflows. Independently authored tenants. One deterministic score.

The workflow archetypes, tool shapes and data shapes come from real ERP operations observed through Nario's dataflywheel; every value in the benchmark is synthetic at production shape. Public Enterprise-Bench, ERP-Bench, APEX and Archipelago materials informed the inspection contract; the corpus, world, verifier and trajectories are new.

Run ERPBench on Harbor ↗Inspect the executable source ↗