Bioinformatics advances · 2026
Critical assessment in the age of AI: challenges and opportunities for computational biology.
Редакционная статьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Schwartz Russell, Zhang Xuegong, Ma Jian, Bromberg Yana, Murphy Robert F, Hunter Lawrence E, Parida Laxmi, Saez-Rodriguez Julio, Stolovitzky Gustavo, Pop Mihai, Ranganathan Shoba, Slonim Donna K, Gaasterland Terry
Аннотация
MOTIVATION: Effective computational tool development depends on our ability to measure progress with rigorous benchmarking, allowing developers to showcase advances and users to select the best of the available options. Computational biology has developed a robust infrastructure of benchmarking standards, tools/resources, and best practices to allow for rigorous and unbiased assessment. This infrastructure, however, faces growing challenges as modern artificial intelligence (AI) methods move to the forefront of computational analysis. The field has imperfectly adapted to issues problematic for benchmarking AI methods, including data entanglement, data leakage, problem-specific performance, limited theoretical guarantees, elusive ground-truth, opaque and often closed models that make reproducibility of results difficult, and dependence on impermanent data sources.
RESULTS: In this paper, we consider how the field should tackle benchmarking in the AI era. We revisit why benchmarking matters and its history in computational biology, examine emerging AI-specific challenges, and outline practical paths forward. We argue that benchmarks should be treated as long-lived community infrastructure rather than one-off artifacts bundled with individual papers. We envision a future of critical assessment i.e. dynamic and responsive to evolving use cases, sustaining progress for advancing science in the age of AI.
AVAILABILITY AND IMPLEMENTATION: N/A.
RESULTS: In this paper, we consider how the field should tackle benchmarking in the AI era. We revisit why benchmarking matters and its history in computational biology, examine emerging AI-specific challenges, and outline practical paths forward. We argue that benchmarks should be treated as long-lived community infrastructure rather than one-off artifacts bundled with individual papers. We envision a future of critical assessment i.e. dynamic and responsive to evolving use cases, sustaining progress for advancing science in the age of AI.
AVAILABILITY AND IMPLEMENTATION: N/A.