SRE World Dataset — Every Task and Model Run

Every task and model run

Pick a task for its ticket, levers, answer key and why each other incident is in or out. Pick a run for every step the model took. The address bar keeps what you open, so links can be shared.

206 of 206 tasks

Pick a task to see its ticket, the levers behind it, the answer key, why every other incident is in or out, and the model runs on it.

The hard tasks, four frontier models

Task	Claude Fable 5.1	Claude Opus 5.5	GPT-6 Astra	GPT-6.1 Sol
P2 “users are reporting site issues”	1.00	1.00	0.33 0.33	0.33 0.33
P5 “support staff reports problems with payments”	0.67	0.67	0.67 0.67	0.67 0.67
P7 “finance staff reports shipping charges look wrong”	1.00	1.00	0.50 0.50	0.50 0.50
P8 “staff are reporting business issues”	0.50	0.50	0.40 0.40	0.40 0.40
P12 “users can't finish buying things”	0.33	1.00	0.33	0.33 0.33
P13 “users can't finish buying things” · Grafana only	0.33	1.00	0.33	0.33
G2 “support staff reports problems with payments” · Grafana only	1.00	0.00	1.00	0.00
G3 “visitors cannot add things to cart” · Grafana only	—	—	1.00	1.00