1. Introduction
Radiomics represents the image-based equivalent of biological omics analyses (e.g., transcriptomics and proteomics), which promises to offer high-fidelity analysis of images for precision medicine purposes [ 1 ]. Analysis of regions of interest in medical images can be performed by extracting radiomics features, which can be used to construct machine learning models that achieve a precise diagnosis, treatment response prediction, and disease prognosis [ 2 , 3 ]. Radiomics has been introduced as a promising image analysis method equivalent to other omics analyses aiming to achieve an image-based biopsy of regions of interest. Nonetheless, radiomics has yet to reach clinical practice but represents a promising research tool for developing predictive image-based signatures that can assist in the diagnosis and treatment of various diseases [ 4 , 5 ]. The quality of radiomics studies can be assessed with the use of the radiomics quality score (RQS) [ 6 , 7 ], which scores radiomics research against a series of standards, including but not limited to the quality of reporting, segmentation, feature extraction, feature selection, the calibration and validation of machine learning models, and the provision of open access data.
Multiple myeloma (MM) is one of the most common hematological malignancies, characterized by osteolytic lesions [ 8 ]. The heterogeneous nature of the disease complicates its diagnosis and treatment, and the similarities of MM lesions relative to osteolytic metastases during imaging complicate the image-based diagnosis of the disease [ 9 ]. The diagnostic approach of MM is based on whole-body CT, MRI, and PET-CT [ 10 ]. A series of radiomics research papers utilizing images from these modalities have been published in an attempt to achieve the differentiation of MM from metastases, prediction of treatment response, identification of molecular subgroups of MM, and analysis of patient survival.
The aims of this systematic review were (a) to provide a comprehensive analysis of the applications of radiomics in MM research and (b) to score manuscripts using RQS to benchmark their quality against existing standards.
2. Materials and Methods
The protocol of this study has been registered with the PROSPERO international register for systematic reviews ( https://www.crd.york.ac.uk/prospero/ , Record ID CRD42023409189, accessed 8 June 2023). Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines were used to prepare this manuscript [ 11 ].
2.1. Database Search Strategy and Selection of Relevant Studies
The search for relevant papers was performed between 1 January 2010 and 1 April 2023 in three databases (dPubMed, Scopus, and Web of Science). The search was performed with the strings “multiple myeloma”, “radiomics”, and “texture”. The detailed strings used in each of the three databases can be found in Supplementary File S1 . Studies from all three databases were collected, and duplicate studies were excluded from further analysis. Three radiologists (MEK, MT, and DL) with 5, 2, and 2 years of experience in radiomics research examined the abstracts of all studies to exclude those that were not eligible: (1) review papers, (2) abstracts and conference papers, (3) editorials, (4) non-English papers. The records were further screened to exclude manuscripts that did not perform radiomics analysis.
2.2. Literature Data Extraction
From each included study, specific information was extracted, including author names, country of origin, year of publication, journal, number of patients, the purpose of the study, imaging modality used, software for radiomics analysis, and the type of MM. Each journal’s impact factor was recorded according to the 2021 Journal Citation Reports (Clarivate) and the journal’s quartile according to Scimago Journal and Country Rank ( https://www.scimagojr.com , accessed 20 April 2023).
2.3. Radiomics Quality Score (RQS)
The quality of radiomics research presented in each study was evaluated by three readers blinded to the results of each other using RQS [ 6 , 7 ]. The first and most experienced reader (R1–MEK) had 5 years of experience in radiomics research and 10 years of experience in medical imaging research, whereas the second and third (R2 and R3—MT and DL) had 2 years of experience in radiomics and 3 years of experience in medical imaging research. Before the initiation of scoring, a training session was performed using radiomics papers unrelated to MM to ensure that manuscripts were scored in precisely the same manner by all readers. Each reader was blinded to the scores that the rest of the readers gave. The RQS consists of 16 items, with a total score ranging between −8 and 36. A total percentage (out of 36) was also calculated for each manuscript.
2.4. Statistical Analysis
Statistical analysis was performed using SPSS v 29 (IBM SPSS for Mac, Armonk, NY, USA). Variables are expressed as frequencies and percentages (categorical) or medians with interquartile ranges (continuous). An adherence metric was calculated to assess adherence to RQS by awarding one point if the authors had gained the minimum points for each RQS item. Data normality was assessed using the Shapiro–Wilk test. RQS comparisons between groups of studies with different characteristics (impact factor, year of publication, journal quartile, and modality) were performed using Mann–Whitney U or Kruskall–Wallis tests according to the number of groups. Agreement between readers was evaluated using the intraclass correlation coefficient (ICC) assessed for absolute agreement, considering acceptable ICC with values >0.75 (good: 0.75–0.9, excellent > 0.9). Statistical significance was defined with α set at 0.05.
3. Results
3.1. Study Selection
After excluding duplicate entries (n = 30), 101 records from all three databases were screened. A total of 21 studies were excluded due to being an inappropriate manuscript type (non-English, review, conference paper, and editorial/letter to the editor), and 57 studies were excluded because of a non-radiomics study design (studies that did not extract radiomics features). This yielded a final sum of n = 23 manuscripts, which were included for analysis ( Figure 1 ).
3.2. Analysis of Included Studies
Detailed characteristics of the 23 included studies can be found in Table 1 . In total, 8 out of 23 studies (34.8%) were published in radiology journals, 4/23 (17.4%) were published in oncological journals, 3/23 (13%) were published in nuclear medicine journals. More than half of the studies (12/23) were published in journals with an Impact Factor > 5. Most studies were published in Q1 journals (15/23—65.2%), with five studies in Q2 (21.7%), two studies in Q3, and one study in a Q4 journal. No study was published before 2019, whereas nine studies were published in each of the years 2021 and 2022. China and Germany were the main countries of origin for published studies, with 34.8% and 21.7% of the studies originating from each of the two, respectively.
The methodological details of individual studies are presented in Table 2 . A total of 2682 patients were examined in the 23 included studies. Several radiomics applications on multiple myeloma have been described in the studies included, with the majority (6 out of 23 studies—26.1%) aimed at differentiating between MM and metastases and the same proportion aimed at predicting treatment response or prognosis. Only 4/23 studies aimed at diagnosing MM compared to normal bone marrow or other hematological conditions. Three out of twenty-three studies evaluated the pattern of infiltration of the bone marrow. In contrast, other miscellaneous applications such as the disease’s load prediction and the presence of high-risk cytogenetic abnormalities represented the minority of the literature (1–2 studies for each application). One of the studies aimed at evaluating the technical reproducibility of radiomics in patients with MM. Almost half of the studies utilized MRI (11/23—47.8%), with PET-CT and CT (single or dual-energy) being the second-most common modalities (6/23 studies each—26.1%). Various commercial and free software was used; a detailed list can be found in Table 2 . Finally, most studies did not define which specific MM type was examined.
3.3. Radiomics Quality Score (RQS)
Adherence to RQS items varied significantly. As shown in Figure 2 , 96% of manuscripts contained some discrimination statistics (e.g., AUC with 95% confidence intervals), and more than 85% (87%) of papers included a sufficiently detailed imaging protocol. It is important that 13% of the studies did not attempt any validation (including internal testing). In contrast, only 9% of the studies included calibration statistics for the presented machine learning models. Feature reduction to account for the possibility of overfitting was performed in 83% of manuscripts, and 57% of studies established the ground truth with the use of a gold standard (e.g., bone marrow biopsy). Finally, only 4% of manuscripts provided either an open access code or open access data, whereas no manuscript included a cost-effectiveness analysis or phantom standardization of radiomics features.
The total RQS for each of the studies for all three readers is presented in Table 3 . The median RQS was 10 for R1 (IQR 5.5–12) and R3 (IQR 8.3–12) and 11 (IQR 7.5–12.5) for R2. The agreement between readers was good (almost excellent), with an ICC of 0.851 (95% CI 0.7 to 0.932). Subgroup analysis showed that there are no statistically significant differences between RQS for papers published in journals with a high (>5) and low impact factor ( Figure 3 and Table 4 ), between papers published before or after 2021, or papers in high or low quartiles or papers dealing with different imaging modalities ( Table 4 ). Impact factor analysis was repeated by removing the only methodological study (which does not contain a predictive model) from our group [ 32 ], noting that there was still no significant correlation between the impact factor and the RQS score ( p > 0.05 for all readers).
4. Discussion
Herein, we presented a comprehensive analysis of radiomics studies on MM. We analyzed study characteristics and demonstrated that the quality of radiomics studies published for the evaluation of MM is inadequate. Importantly, we demonstrated low adherence rates to most RQS items and a low total RQS, and we showed that low quality is generic and not specific to the journal’s characteristics. This finding is extremely important in evaluating published research and highlights the need to conduct high-quality research in the field.
The issue of inadequate-quality published radiomics research has been recently highlighted [ 5 ] in studies dealing with various topics [ 35 , 36 ], strongly indicating a potential lack of reproducibility. Most examined studies did not include external validation of their results, with two of them also skipping testing using internal data. The lack of external validation significantly limits the generalization capacity of machine learning models and is a common problem in published papers on AI using medical images, where only 6–10% of published studies have been tested on an external dataset [ 37 ]. This is of high importance since algorithm performance is consistently lower when validated on external datasets and even lower in “real-life” conditions [ 37 , 38 ]. Our findings signify the fact that even though the published algorithms may have the potential to revolutionize MM diagnosis and management, the lack of external validation hinders their adoption, reducing the trust in the results.
RQS is, at the moment, the most important tool for evaluating radiomics research. Adherence to RQS items is supposed to indicate high-quality research. In our case, some items were either addressed in none or a very limited subset of the studies, including the use of phantoms, cost-effectiveness analysis, and the provision of open access data. The lack of open access data may be attributed to limitations related to the publication of patient data. However, anonymized radiomics numerical values do not fall under this category. The publication of such studies indicates a lack of robust reviewing practices and insufficient journal and reviewer expertise in radiomics and machine learning. The rapid increase in machine learning publications in musculoskeletal imaging [ 39 ] has caused an increased demand for expert reviewers. Thus, assigning such manuscripts to inexperienced reviewers with potential expertise in musculoskeletal or hematological malignancies can lead to the publication of low-quality research, even in high-impact journals. As indicated by our results, there was no difference between high- and low-impact journals regarding their RQS score. To overcome this problem, guides for radiomics research have been published [ 1 , 3 ], providing basic directions to reviewers; moreover, journals need to adhere to the basic standards of RQS. It also needs to be pointed out that RQS has not been specifically designed to evaluate studies that do not contain predictive models. Therefore, technical radiomics studies may receive a lower RQS score because they present no predictive models. Our study included one such technical manuscript by Wennmann et al. [ 31 ], which received a median RQS of 12. This is almost identical to the median RQS of all studies in our sample; therefore, it was not analyzed separately since it does not negatively affect the results of the study. However, it needs to be pointed out that such studies can be of higher quality than the quality indicated by their RQS score since they are negatively scored for the absence of a predictive model that they were not supposed to analyze.
Radiomics, a rapidly evolving field within medical imaging, focuses on extracting high-dimensional quantitative data from medical images, utilizing these data to uncover hidden information that is not readily discernible by the human eye [ 1 ]. This innovative approach presents a promising frontier for diagnosing MM and the differentiation between MM and lytic metastases. The complex nature of MM lesions, coupled with their variable appearance in imaging, creates a pressing need for advanced diagnostic techniques that enable accurate, efficient, and personalized interventions for affected patients. MM is a hematologic malignancy characterized by lytic bone lesions, making their distinction from lytic metastases caused by other types of malignancy a challenging task for clinicians [ 10 ]. Accurate diagnosis and differentiation are crucial as the treatment strategies, prognoses, and overall management of MM patients are markedly different from those with metastatic bone disease. Radiomics, by leveraging advanced machine learning algorithms and computational models, has the potential to identify unique, subvisual patterns and features within MM-affected bones that can accurately distinguish them from lytic metastases. Radiomics features provide a comprehensive, quantitative analysis of tumor characteristics, which can improve diagnostic accuracy and potentially reveal novel imaging biomarkers for MM [ 3 , 4 ]. Consequently, these advancements may lead to earlier and more personalized interventions for patients with multiple myeloma, ultimately enhancing their prognosis and quality of life.
One of the most important reasons that radiomics has not yet found extensive applications on MM is the nature of MM lesions, which are disseminated across the bone marrow, presenting an important challenge with regard to the segmentation of the entire tumor load given that this is not evenly distributed across the skeleton [ 40 ] and the mutations associated with the lesions can be spatially heterogeneous [ 41 ]. Therefore, in order to implement radiomics in clinical practice for MM patients, methods that allow the segmentation of multiple focal lesions and extended areas of diffuse infiltration or whole bones are required since the manual segmentation of this scale is extremely tedious and potentially unreliable. Methods such as the atlas-based semi-automatic segmentation of whole-body diffusion-weighted imaging and deep learning applications combined with radiomics have already been proposed [ 31 , 42 , 43 , 44 ] and may be the solution to the future translation of radiomics research to the clinic.
Several medical imaging modalities, including computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET), can be utilized for radiomic analysis in MM patients. Our work showed that almost half of the manuscripts utilized MRI-based radiomics. CT-derived radiomics features may capture structural alterations in the trabecular bone, while MRI-based radiomics can offer insights into the tissue composition in the bone marrow. PET-based radiomics, on the other hand, can evaluate the metabolic activity of MM lesions and assess their response to therapy [ 8 , 10 ]. By integrating multi-modal imaging data, radiomics offers a comprehensive view of MM lesions, potentially enabling more accurate and precise diagnoses. The future integration of radiomics with other omics data, such as genomics, proteomics, and metabolomics, could create a holistic understanding of MM, uncovering complex relationships between imaging phenotypes and the underlying molecular mechanisms driving disease progression. This integration, often referred to as radiogenomics or radiotranscriptomics [ 45 , 46 ], can pave the way for personalized medicine approaches in MM, guiding clinicians in tailoring treatment plans based on the specific characteristics of each patient’s disease. Utilizing such integration methods for the diagnosis of MM would necessitate high-quality radiomics methods that are not currently available in the MM literature. The development of high-quality radiomics signatures for implementation in the clinical management of multiple myeloma requires interdisciplinary collaboration among radiologists, oncologists, physicists, data scientists, and bioinformaticians. This collaborative approach can foster the development of novel radiomic models, drive the discovery of new imaging biomarkers, and ultimately contribute to the improvement of diagnostic accuracy, treatment planning, and overall patient outcomes.
Our study has certain strengths and limitations. The systematic review of the literature and the comprehensive analysis of study characteristics represent the strengths of our study. Another important strength is the high ICC between the three readers. Such high agreement between readers has been reported in other RQS assessment studies [ 47 ]. Nonetheless, one limitation of our study is related to the inherent limitations of RQS. The score itself is empirical and was not validated when first published by Lambin et al. [ 4 ]. Therefore, it includes items, such as the use of phantoms for feature standardization and cost-effectiveness analysis, that are consistently not found in any published radiomics papers, even the ones that adhere to the highest quality standards [ 36 , 47 , 48 ]. Thus, it has been suggested that these items could unnecessarily reduce the total RQS, preventing studies from obtaining the maximum number (36) of points. Nonetheless, in our study, the maximum total RQS was 20, which cannot be compensated by scoring points in the items mentioned above. In similar RQS systematic reviews for other types of disease, the maximum RQS score was similar. The same is true for the median RQS value, which was found to be 10 in our study, comparable to values shown by other published systematic reviews. Such examples include a maximum of 22 in studies of hepatocellular carcinoma [ 49 ] and a maximum of 16 in studies of ischemic strokes [ 50 ].
5. Conclusions
In conclusion, radiomics presents a powerful and innovative approach for evaluating MM. By unlocking the hidden information within medical images, radiomics has the potential to revolutionize MM diagnosis, risk stratification, and treatment planning. Our study has highlighted the low scientific quality of radiomics papers related to MM, similarly to other fields of radiomics research, demonstrating that the methodological limitations of existing studies are not related to bibliometric (journal, impact factor, quartile, etc.) data. These findings emphasize the need to tighten current publication standards in order to publish radiomics research studies of high quality.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Supplementary Materials
The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/diagnostics13122021/s1 . Supplementary File S1: Full strings used for database search.
Click here for additional data file.
Author Contributions
Conceptualization, M.E.K.; methodology, M.E.K.; software, M.E.K.; validation, M.E.K. and A.H.K.; formal analysis, M.E.K.; investigation, M.T., D.L., E.K., G.K. and A.T.; data curation, M.E.K., D.L. and M.T.; writing—original draft preparation, M.E.K., M.T., D.L., E.K., G.K., A.T. and A.H.K.; writing—review and editing, M.E.K., M.T., D.L., E.K., G.K., A.T. and A.H.K.; visualization, M.E.K., M.T., D.L., E.K., G.K., A.T. and A.H.K.; supervision, M.E.K. and A.H.K. All authors have read and agreed to the published version of the manuscript.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
All data required to reproduce results are provided in the paper.
Conflicts of Interest
The authors declare no conflict of interest.
References
1. van Timmeren J.E. Cester D. Tanadini-Lang S. Alkadhi H. Baessler B. Radiomics in medical imaging—“How-to” guide and critical reflection Insights Imaging 2020 11 91 10.1186/s13244-020-00887-2 32785796 PMC7423816
2. Mühlbauer J. Egen L. Kowalewski K.-F. Grilli M. Walach M.T. Westhoff N. Nuhn P. Laqua F.C. Baessler B. Kriegmair M.C. Radiomics in renal cell carcinoma—A systematic review and meta-analysis Cancers 2021 13 1348 10.3390/cancers13061348 33802699 PMC8002585
3. Papanikolaou N. Matos C. Koh D.M. How to develop a meaningful radiomic signature for clinical use in oncologic patients Cancer Imaging 2020 20 33 10.1186/s40644-020-00311-4 32357923 PMC7195800
4. Gillies R.J. Kinahan P.E. Hricak H. Radiomics: Images are more than pictures, they are data Radiology 2016 278 563 577 10.1148/radiol.2015151169 26579733 PMC4734157
5. Pinto Dos Santos D. Dietzel M. Baessler B. A decade of radiomics research: Are images really data or just patterns in the noise? Eur. Radiol. 2021 31 1 4 10.1007/s00330-020-07108-w 32767103 PMC7755615
6. Lambin P. Leijenaar R.T.H. Deist T.M. Peerlings J. de Jong E.E.C. van Timmeren J. Sanduleanu S. Larue R.T.H.M. Even A.J.G. Jochems A. Radiomics: The bridge between medical imaging and personalized medicine Nat. Rev. Clin. Oncol. 2017 14 749 762 10.1038/nrclinonc.2017.141 28975929
7. Klontzas M.E. Gatti A.A. Tejani A.S. Kahn C.E. AI reporting guidelines: How to select the best one for your research Radiol. Artif. Intell. 2023 5 e230055 10.1148/ryai.230055 37293341 PMC10245184
8. Healy C.F. Murray J.G. Eustace S.J. Madewell J. O’Gorman P.J. O’Sullivan P. Multiple myeloma: A review of imaging features and radiological techniques Bone Marrow Res. 2011 2011 583439 10.1155/2011/583439 22046568 PMC3200072
9. Sandal R. Mishra K. Jandial A. Khadwal A. Malhotra P. Multiple myeloma and pepperpot skull QJM 2018 111 917 10.1093/qjmed/hcy165 30020515
10. Filho A.G.O. Carneiro B.C. Pastore D. Silva I.P. Yamashita S.R. Consolo F.D. Hungria V.T.M. Sandes A.F. Rizzatti E.G. Nico M.A.C. Whole-body imaging of multiple myeloma: Diagnostic criteria RadioGraphics 2019 39 1077 1097 10.1148/rg.2019180096 31283452
11. Page M.J. McKenzie J.E. Bossuyt P.M. Boutron I. Hoffmann T.C. Mulrow C.D. Shamseer L. Tetzlaff J.M. Akl E.A. Brennan S.E. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews PLoS Med 2021 18 e1003583 10.1371/journal.pmed.1003583 33780438 PMC8007028
12. Chen K. Cao J. Zhang X. Wang X. Zhao X. Li Q. Chen S. Wang P. Liu T. Du J. Differentiation between spinal multiple myeloma and metastases originated from lung using multi-view attention-guided network Front. Oncol. 2022 12 981767 10.3389/fonc.2022.981769 36158659 PMC9495278
13. Ekert K. Hinterleitner C. Baumgartner K. Fritz J. Horger M. Extended texture analysis of non-enhanced whole-body MRI image data for response assessment in multiple myeloma patients undergoing systemic therapy Cancers 2020 12 761 10.3390/cancers12030761 32213834 PMC7140042
14. Hwang E.-J. Jung J.-Y. Lee S.K. Lee S.E. Jee W.H. Machine learning for diagnosis of hematologic diseases in magnetic resonance imaging of lumbar spines Sci. Rep. 2019 9 6046 10.1038/s41598-019-42579-y 30988360 PMC6465258
15. Jamet B. Morvan L. Nanni C. Michaud A.V. Bailly C. Chauvie S. Moreau P. Touzeau C. Zamagni E. Bodet-Milin C. Random survival forest to predict transplant-eligible newly diagnosed multiple myeloma outcome including FDG-PET radiomics: A combined analysis of two independent prospective European trials Eur. J. Nucl. Med. Mol. Imag. 2021 48 1005 1015 10.1007/s00259-020-05049-6 33006656
16. Jin Z. Wang Y. Wang Y. Mao Y. Zhang F. Yu J. Application of 18F-FDG PET-CT images based radiomics in identifying vertebral multiple myeloma and bone metastases Front. Med. 2022 9 874847 10.3389/fmed.2022.874847 35510246 PMC9058063
17. Lee S. Lee S.Y. Kim S. Huh Y.J. Lee J. Lee K.E. Jung J.Y. Differentiating multiple myeloma and osteolytic bone metastases on contrast-enhanced computed tomography scans: The feasibility of radiomics analysis Diagnostics 2023 13 755 10.3390/diagnostics13040755 36832243 PMC9955828
18. Li Y. Liu Y. Yin P. Hao C. Hao C. Sun C. Chen L. Wang S. Hong N. MRI-based bone marrow radiomics nomogram for prediction of overall survival in patients with multiple myeloma Front. Oncol. 2021 11 709813 10.3389/fonc.2021.709813 34926240 PMC8671997
19. Li Y. Yin P. Liu Y. Hao C. Chen L. Sun C. Wang S. Hong N. Radiomics models based on magnetic resonance imaging for prediction of the response to bortezomib-based therapy in patients with multiple myeloma BioMed Res. Int. 2022 2022 6911246 10.1155/2022/6911246 36105939 PMC9467708
20. Liu J. Guo W. Zeng P. Geng Y. Liu Y. Ouyang H. Lang N. Yuan H. Vertebral MRI-based radiomics model to differentiate multiple myeloma from metastases: Influence of features number on logistic regression model performance Eur. Radiol. 2022 32 572 581 10.1007/s00330-021-08150-y 34255157
21. Liu J. Wang C. Guo W. Zeng P. Liu Y. Lang N. Yuan H. A preliminary study using spinal MRI-based radiomics to predict high-risk cytogenetic abnormalities in multiple myeloma Radiol. Med. 2021 126 1226 1235 10.1007/s11547-021-01388-y 34159496
22. Mannam P. Murali A. Gokulakrishnan P. Venkatachalapathy E. Venkata Sai P. Radiomic analysis of positron-emission tomography and computed tomography images to differentiate between multiple myeloma and skeletal metastases Indian J. Nucl. Med. 2022 37 217 226 10.4103/ijnm.ijnm_111_21 36686312 PMC9855237
23. Mesguich C. Hindie E. de Senneville B.D. Tlili G. Pinaquy J.-B. Marit G. Saut O. Improved 18-FDG PET/CT diagnosis of multiple myeloma diffuse disease by radiomics analysis Nucl. Med. Commun. 2021 42 1135 1143 10.1097/MNM.0000000000001437 34001823
24. Milara E. Gómez-Grande A. Tomás-Soler S. Seiffert A.P. Alonso R. Gómez E.J. Martínez-López J. Sánchez-González P. Bone marrow segmentation and radiomics analysis of [18F]FDG PET/CT images for measurable residual disease assessment in multiple myeloma Comput. Methods Progr. Biomed. 2022 225 107083 10.1016/j.cmpb.2022.107083 36044803
25. Park H. Lee S.-Y. Lee J. Pak J. Lee K. Lee S.E. Jung J.Y. Detecting multiple myeloma infiltration of the bone marrow on CT scans in patients with osteopenia: Feasibility of radiomics analysis Diagnostics 2022 12 923 10.3390/diagnostics12040923 35453971 PMC9025143
26. Reinert C.P. Krieg E. Esser M. Nikolaou K. Bösmüller H. Horger M. Role of computed tomography texture analysis using dual-energy-based bone marrow imaging for multiple myeloma characterization: Comparison with histology and established serologic parameters Eur. Radiol. 2021 31 2357 2367 10.1007/s00330-020-07320-8 33011876 PMC7979667
27. Reinert C.P. Krieg E.M. Bösmüller H. Horger M. Mid-term response assessment in multiple myeloma using a texture analysis approach on dual energy-CT-derived bone marrow images—A proof of principle study Eur. J. Radiol. 2020 131 109214 10.1016/j.ejrad.2020.109214 32835853
28. Ripani D. Caldarella C. Za T. Rossi E. de Stefano V. Giordano A. Progression to symptomatic multiple myeloma predicted by texture analysis-derived parameters in patients without focal disease at 18 F-FDG PET/CT Clin. Lymph. Myel. Leuk. 2021 21 536 544 10.1016/j.clml.2021.03.014 33985932
29. Schenone D. Dominietto A. Campi C. Frassoni F. Cea M. Aquino S. Angelucci E. Rossi F. Torri L. Bignotti B. Radiomics and artificial intelligence for outcome prediction in multiple myeloma patients undergoing autologous transplantation: A feasibility study with ct data Diagnostics 2021 11 1759 10.3390/diagnostics11101759 34679456 PMC8535117
30. Tagliafico A.S. Cea M. Rossi F. Valdora F. Bignotti B. Succio G. Gualco S. Conte A. Dominietto A. Differentiating diffuse from focal pattern on computed tomography in multiple myeloma: Added value of a radiomics approach Eur. J. Radiol. 2019 121 108739 10.1016/j.ejrad.2019.108739 31733431
31. Wennmann M. Klein A. Bauer F. Chmelik J. Grözinger M. Uhlenbrock C. Lochner J. Nonnenmacher T. Rotkopf L.T. Sauer S. Combining deep learning and radiomics for automated, objective, comprehensive bone marrow characterization from whole-body MRI: A multicentric feasibility study Investig. Radiol. 2022 57 752 763 10.1097/RLI.0000000000000891 35640004
32. Wennmann M. Bauer F. Klein A. Chmelik J. Grözinger M. Rotkopf L.T. Neher P. Gnirs R. Kurz F.T. Nonnenmacher T. In vivo repeatability and multiscanner reproducibility of MRI radiomics features in patients with monoclonal plasma cell disorders Investig. Radiol. 2023 58 253 264 10.1097/RLI.0000000000000927 36165988
33. Wu Z.M. Bian T.M. Dong C. Duan S. Fei H. Hao D. Xu W. Spinal MRI-based radiomics analysis to predict treatment response in multiple myeloma J. Comput. Assist. Tomogr. 2022 46 447 454 35405690 10.1097/RCT.0000000000001298
34. Xiong X. Wang J. Hu S. Dai Y. Zhang Y. Hu C. Differentiating between multiple myeloma and metastasis subtypes of lumbar vertebra lesions using machine learning–based radiomics Front. Oncol. 2021 11 601699 10.3389/fonc.2021.601699 33718148 PMC7943866
35. Ponsiglione A. Stanzione A. Spadarella G. Baran A. Cappellini L.A. Lipman K.G. van Ooijen P. Cuocolo R. Ovarian imaging radiomics quality score assessment: An EuSoMII radiomics auditing group initiative Eur. Radiol. 2023 33 2239 2247 10.1007/s00330-022-09180-w 36303093 PMC9935717
36. Cannella R. Vernuccio F. Klontzas M.E. Ponsiglione A. Petrash E. Ugga L. Pinto Dos Santos D. Cuocolo R. Systematic review with radiomics quality score of cholangiocarcinoma: An EuSoMII Radiomics Auditing Group Initiative Insights Imaging 2023 14 21 36720726 10.1186/s13244-023-01365-1 PMC9889586
37. Yu A.C. Mohajer B. Eng J. External validation of deep learning algorithms for radiologic diagnosis: A systematic review Radiol. Artif. Intell. 2022 4 e210064 10.1148/ryai.210064 35652114 PMC9152694
38. Oakden-Rayner L. Gale W. Bonham T.A. Lungren M.P. Carneiro G. Bradley A.P. Palmer L.J. Validation and algorithmic audit of a deep learning system for the detection of proximal femoral fractures in patients in the emergency department: A diagnostic accuracy study Lancet Digit. Health 2022 4 e351 e358 10.1016/S2589-7500(22)00004-8 35396184
39. Klontzas M.E. Papadakis G.Z. Marias K. Karantanas A.H. Musculoskeletal trauma imaging in the era of novel molecular methods and artificial intelligence Injury 2020 51 2748 2756 10.1016/j.injury.2020.09.019 32972725
40. Latifoltojar A. Boyd K. Riddell A. Kaiser M. Messiou C. Characterising spatial heterogeneity of multiple myeloma in high resolution by whole body magnetic resonance imaging: Towards macro-phenotype driven patient management Magn. Reson. Imaging 2021 75 60 64 10.1016/j.mri.2020.10.005 33075451
41. Rasche L. Chavan S.S. Stephens O.W. Patel P.H. Tytarenko R. Ashby C. Bauer M. Stein C. Deshpande S. Wardell C. Spatial genomic heterogeneity in multiple myeloma revealed by multi-region sequencing Nat. Commun. 2017 8 268 10.1038/s41467-017-00296-y 28814763 PMC5559527
42. Wennmann M. Neher P. Stanczyk N. Kahl K.-C. Kächele J. Weru V. Hielscher T. Grözinger M. Chmelik J. Zhang K.S. Deep learning for automatic bone marrow apparent diffusion coefficient measurements from whole-body magnetic resonance imaging in patients with multiple myeloma: A retrospective multicenter study Investig. Radiol. 2023 58 273 282 10.1097/RLI.0000000000000932 36256790
43. Almeida S.D. Santinha J. Oliveira F.P.M. Ip J. Lisitskaya M. Lourenço J. Uysal A. Matos C. João C. Papanikolaou N. Quantification of tumor burden in multiple myeloma by atlas-based semi-automatic segmentation of WB-DWI Cancer Imaging 2020 20 6 10.1186/s40644-020-0286-5 31931880 PMC6958755
44. Klein A. Warszawski J. Hillengaß J. Maier-Hein K.H. Automatic bone segmentation in whole-body CT images Int. J. Comput. Assist. Radiol. Surg. 2019 14 21 29 10.1007/s11548-018-1883-7 30426400
45. Oikonomou E.K. Williams M.C. Kotanidis C.P. Desai M.Y. Marwan M. Antonopoulos A.S. Thomas K.E. Thomas S. Akoumianakis I. Fan L.M. A novel machine learning-derived radiotranscriptomic signature of perivascular fat improves cardiac risk prediction using coronary CTangiography Eur. Heart J. 2019 40 3529 3543 10.1093/eurheartj/ehz592 31504423 PMC6855141
46. Trivizakis E. Souglakos I. Karantanas A.H. Marias K. Deep radiotranscriptomics of non-small cell lung carcinoma for assessing molecular and histology subtypes with a data-driven analysis Diagnostics 2021 11 2383 10.3390/diagnostics11122383 34943617 PMC8700168
47. Spadarella G. Calareso G. Garanzini E. Ugga L. Cuocolo A. Cuocolo R. MRI based radiomics in nasopharyngeal cancer: Systematic review and perspectives using radiomic quality score (RQS) assessment Eur. J. Radiol. 2021 140 109744 10.1016/j.ejrad.2021.109744 33962253
48. Ugga L. Perillo T. Cuocolo R. Stanzione A. Romeo V. Green R. Cantoni V. Brunetti A. Meningioma MRI radiomics and machine learning: Systematic review, quality score assessment, and meta-analysis Neuroradiology 2021 63 1293 1304 10.1007/s00234-021-02668-0 33649882 PMC8295153
49. Yuan E. Chen Y. Song B. Quality of radiomics for predicting microvascular invasion in hepatocellular carcinoma: A systematic review Eur. Radiol. 2023 33 3467 3477 10.1007/s00330-023-09414-5 36749371
50. Dragoș H.M. Stan A. Pintican R. Feier D. Lebovici A. Panaitescu P.-Ș. Dina C. Strilciuc S. Muresanu D.F. MRI radiomics and predictive models in assessing ischemic stroke outcome—A systematic review Diagnostics 2023 13 857 10.3390/diagnostics13050857 36900001 PMC10000411