METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII.
Обработка статьи · этапы, ошибки и управление
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Kocak Burak, Akinci D'Antonoli Tugba, Mercaldo Nathaniel, Alberich-Bayarri Angel, Baessler Bettina, Ambrosini Ilaria, Andreychenko Anna E, Bakas Spyridon, Beets-Tan Regina G H, Bressem Keno, Buvat Irene, Cannella Roberto, Cappellini Luca Alessandro, Cavallo Armando Ugo, Chepelev Leonid L, Chu Linda Chi Hang, Demircioglu Aydin, deSouza Nandita M, Dietzel Matthias, Fanni Salvatore Claudio, Fedorov Andrey, Fournier Laure S, Giannini Valentina, Girometti Rossano, Groot Lipman Kevin B W, Kalarakis Georgios, Kelly Brendan S, Klontzas Michail E, Koh Dow-Mu, Kotter Elmar, Lee Ho Yun, Maas Mario, Marti-Bonmati Luis, Müller Henning, Obuchowski Nancy, Orlhac Fanny, Papanikolaou Nikolaos, Petrash Ekaterina, Pfaehler Elisabeth, Pinto Dos Santos Daniel, Ponsiglione Andrea, Sabater Sebastià, Sardanelli Francesco, Seeböck Philipp, Sijtsema Nanna M, Stanzione Arnaldo, Traverso Alberto, Ugga Lorenzo, Vallières Martin, van Dijk Lisanne V, van Griethuysen Joost J M, van Hamersvelt Robbert W, van Ooijen Peter, Vernuccio Federica, Wang Alan, Williams Stuart, Witowski Jan, Zhang Zhongyi, Zwanenburg Alex, Cuocolo Renato
Аннотация
METHODS: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated.
RESULT: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community.
CONCLUSION: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers.
CRITICAL RELEVANCE STATEMENT: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning.
KEY POINTS: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics ).