Retrieval, embeddings and long context / Benchmark leaderboard
RULER
Synthetic long-context retrieval, aggregation and reasoning at varying lengths.
No published company result yet.
This benchmark is in the research directory. Its task package, adapter and grading protocol need qualification before a hosted run can be offered.
Arrange an evaluation for your company →What this benchmark measures
Metrics
Accuracy versus context length and task family.
Execution requirements
Long-context text model.
Scope and limitations
Context-window capacity is not demonstrated simply by accepting a long input; synthetic success also does not establish enterprise document quality.
Evaluation availability
Catalog entry. Request a managed evaluation to qualify your agent interface and the benchmark’s native grading requirements.
Sources and company fit
- Original benchmark ↗Established diagnostic
Companies whose products may fit
Research recommendations based on product capabilities. These companies have not necessarily run this benchmark or integrated with Blobfish.
Compatibility notes for each company
Anthropic: Capability-aligned; adapter/access to qualify
Cohere: Direct component API + selected system adapters
Glean: Capability-aligned; adapter/access to qualify
Google DeepMind: Capability-aligned; adapter/access to qualify
Hebbia: Capability-aligned; adapter/access to qualify
Mistral: Capability-aligned; adapter/access to qualify
OpenAI: Capability-aligned; adapter/access to qualify
Writer: Capability-aligned; adapter/access to qualify