SQL, analytics and data engineering / Benchmark leaderboard
DABstep
450 multi-step data-analysis challenges derived from payments workloads.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Answer/task correctness under the benchmark evaluator.
Execution requirements
Data agent with Python/shell and benchmark files.
Scope and limitations
Different from pure text-to-SQL: requires reasoning over files, documentation and operational definitions.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established specialist
- Harbor · adyen/dabstep ↗450 source tasks · Public · Public detail metadata + family research
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Databricks: Capability-aligned; adapter/access to qualify
Hebbia: Capability-aligned; adapter/access to qualify
Hex: Capability-aligned; adapter/access to qualify
Julius AI: Capability-aligned; adapter/access to qualify
MotherDuck: Capability-aligned; adapter/access to qualify
Rogo: Capability-aligned; adapter/access to qualify
Sigma: Capability-aligned; adapter/access to qualify
Snowflake: Capability-aligned; adapter/access to qualify
ThoughtSpot: Capability-aligned; adapter/access to qualify