Introducing DeepWeb-Bench: A New Benchmark for Challenging Deep Research in Language Models

DeepWeb-Bench is a new deep research benchmark designed to evaluate frontier language models through demanding tasks that require extensive evidence collection, cross-source reconciliation, and long-horizon multi-step derivation. This benchmark introduces four capability families: Retrieval, Derivation, Reasoning, and Calibration, and provides a detailed source-provenance record for each reference answer. Evaluation of nine frontier models reveals that retrieval issues account for only 12-14% of errors, while over 70% of failures stem from derivation and calibration challenges. The findings indicate distinct failure modes between strong and weak models and highlight significant specialization across models. The benchmark's public release includes data, evaluation rubrics, and code for further research.