Проверка основанных на правилах методов обнаружения рецидивов при множественном склерозе.

Background

Medications for the treatment of patients with multiple sclerosis (MS), especially those with relapsing-remitting multiple sclerosis, are approved in pharmaceutical markets primarily based on their efficacy on reducing the risk of relapse [ 1 – 6 ]. A relapse episode may include new, worsened, or recurrent symptoms of neurologic dysfunction that persist for at least 24 h with or without recovery and occur in the absence of fever or infection [ 1 – 6 ]. Several pharmacological interventions have demonstrated efficacy in reducing the annual relapse rate among patients with MS in pivotal Phase III randomized controlled trials [ 1 – 6 ]. Relapses in clinical trials are ascertained using a combination of physical and neurological signs/symptoms, as well as neurological and functional measurement scales (e.g., the Expanded Disability Status Scale [EDSS] and Multiple Sclerosis Functional Composite [MSFC]) [ 1 – 6 ]. However, there is a lack of consensus on a standardized approach to ascertain relapses in real-world settings [ 7 – 10 ]. Additionally, no internationally recognized and validated diagnostic codes exist for relapses, largely due to the variability in the application and documentation of trial-based measurement scales in real-world settings [ 7 – 12 ]. This lack of consensus leads to a gap in the assessment of efficacy vs. effectiveness of pharmacological treatments in clinical trials (i.e., ideal settings) vs. real-world settings [ 8 , 13 , 14 ].

Relapses in real-world effectiveness studies are typically identified using study-specific relapse case definition, or through pre-defined detection methods [ 15 – 18 ]. Few validated detection methods have been developed; however, these often rely on non-electronic health records (EHR) data or information contained within unstructured data formats [ 9 , 10 , 19 – 21 ]. Given the aforementioned absence of validated diagnostic codes for relapses within structured EHRs, and despite the existence of certain MS registries that may capture some relevant details, albeit without consistent linkage exercise across institutions, the development and validation of rule-based detection methods using structured EHR data can enhance the ability to conduct real-world effectiveness studies by enabling relapse detection without reliance on additional data sources [ 12 , 22 ]. Structured EHRs provide standardized and longitudinal records of diagnoses, treatments, and hospital encounters, which improve data completeness and allow for consistent methods application across a large patient population [ 23 , 24 ]. The validation results will also enable researchers to conduct appropriate quantitative bias analyses to adjust for potential measurement bias. Therefore, this study aimed to develop and assess the validity of rule-based methods to detect relapse using longitudinal EHRs from a multi-regional healthcare institution in Saudi Arabia, thereby enhancing the representativeness and generalizability of the findings to broader healthcare settings.

Methods

Study design, data source, and patient population

This study was a retrospective, multicenter validation study. EHR data from a 5,421-bed healthcare institution covering multiple regions in Saudi Arabia were used to identify the study population [ 25 , 26 ]. All adult patients (≥ 18 years old) with at least one MS diagnosis, recorded using the International Classification of Diseases, Tenth Revision [ICD-10-CM: G35], who visited the hospital between 01 January 2016 and 31 December 2023 were identified and included in the study.

Study relapse detection methods

Two rule-based detection methods were developed (Table 1 ). These methods, incorporated updated internationally recognized diagnostic criteria for relapses in MS patients, elements from previously published relapse-detection approaches (Supplementary Table 1 ), and institution-specific recording practices for key diagnostic and treatment components of MS [ 9 , 10 , 21 , 27 ]. The latter were identified through consultations with the study neurologists. Other detection methods were explored during study design phase, including outpatient visits combined with corticosteroid dispensing within 7 days and other temporal criteria. However, these approaches detected substantially fewer potential relapse episodes and were, therefore, not prioritized for validation.

The two methods were as follows:

I. Patients were identified as having experienced a relapse episode according to Detection Method I if they met the following: the presence of a diagnostic code of MS, a prescription for high-dose corticosteroid therapy of at least 1 gram of methylprednisolone or an equivalent corticosteroid, and hospitalization for at least one day [ 9 , 10 , 21 , 27 , 28 ]. High-dose corticosteroid therapy, from 1 g up to 2 g, reflects the standard treatment and the institutional inpatient protocol for the management of acute MS relapses within the participating healthcare institution. Furthermore, this therapy has been widely used as a marker of relapse in both clinical trials and real-world studies [ 5 – 10 , 21 , 27 – 29 ]. Short hospitalizations of one day generally indicate acute neurological deterioration requiring intravenous corticosteroids, consistent with the management of relapse in routine clinical practice [ 5 – 10 , 21 , 27 , 29 ].

II. Detection Method II employed a more flexible approach to identify relapse episodes. It required the presence of at least one MS diagnostic code in combination with either a hospitalization lasting three or more days, or consecutive hospitalizations within a 7-day period with a cumulative duration of at least three days under the care of the neurology department. These thresholds were selected to capture relapse episodes necessitating prolonged inpatient management or readmission due to delayed recovery or complications, while minimizing inclusion of short elective or diagnostic admissions. This reflects patterns observed in clinical practice guidelines and supported by prior studies on relapse-detection approaches [ 5 – 10 , 21 , 27 , 29 , 30 ].

Sampling methods

The developed rule-based methods were applied to the extracted MS sample to detect patients with at least one relapse episode (cases). Following, two sampling methods were used to select cases (patients with relapse) and controls (patients without a relapse), with the aim of enhancing both internal validity and generalizability of the study’s detection methods (shown in Fig. 1 ):

I. Sampling Method I involved selecting a simple random sample of 400 patients from the full cohort of 1,812 patients with at least one MS diagnosis, irrespective of their stratum (detected cases or controls status). Patients were classified as cases if they met the rules of detection methods I or II, and as controls otherwise. No hierarchical prioritization between the detection methods was applied. This approach preserves the original distribution of relapses in the population and allows for the calculation of valid sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV) [ 31 ].

II. Sampling Method II was applied to ensure sufficient number of detected cases for validation of each of the detection methods, as sampling Method I does not allow control over the expected number of cases or controls per cell, which may result in only a few cases being identified by either method. Consequently, two separate samples were created, each including 100 cases detected by one of the detection methods. For each sample, 100 controls were randomly selected from patients who did not meet the criteria of the corresponding detection method under validation and were matched on age and sex. Sampling using this method is likely to alter the marginal prevalence of relapse compared to the original MS population, thereby compromising the validity of sensitivity and specificity. However, the PPVs and NPVs derived from this method remain valid, although not transportable to other populations, as they are unaffected by the sample’s marginal prevalence [ 31 ].

Validation

Sampling Method I yielded 174 cases and 226 controls. Using Sampling Method II, two samples were created for the validation of the two detection methods, with each sample comprising 100 cases and 100 controls (shown in Fig. 1 ). All study samples were manually validated by two study neurologists, who were blinded to the case–control classification derived from the detection methods. They were provided only with encounter dates to determine whether a relapse had occurred within the specified encounter period and, if not, to indicate the nearest relapse date. In cases of discrepancies between the two neurologists’ assessments, the cases were flagged for review by two other neurologists, who also performed random cross-checks of the validations to ensure accuracy. The validation process involved a detailed review of the original patients’ EHRs, including clinician notes, radiology reports, and diagnosis records, to detect evidence of the presence or absence of relapse. Each review was categorized as follows: ‘yes’ for confirmed relapse, ‘no’ for confirmed non-relapse, ‘not MS’ for misdiagnosed patients, and ‘unknown’ for cases with insufficient information to make a determination.

Statistical analysis

Descriptive statistics were reported for patient characteristics, including variables (age, sex, and regional distribution) and comorbidities at the time of MS diagnosis. Based on a comparison between the output from each detection method and the manual EHR review results, sensitivity, specificity, PPV, and NPV were calculated with their corresponding 95% confidence intervals (CIs) [ 32 , 33 ]. Patients adjudicated as “Not MS” or “Unknown” during manual review were retained in the validation process to reflect real-world data conditions and to avoid overestimating performance metrics. All statistical analysis was conducted using RStudio software (version 4.4.0) with a p -value threshold of 0.05 for statistical significance.

Results

A total of 1,812 patients, with a mean age of 37 ± 12 years, were identified as having at least one MS diagnosis (ICD-10-CM: G35) during the study period, of whom two-thirds were females (66%). Detection Method I identified 499 (27.5%) patients with at least one relapse episode, yielding a total of 800 episodes. Detection Method II identified 594 patients (32.8%) with at least one relapse episode and captured 1,068 episodes (Table 2 ) (Fig. 1 ). At the episode level, 573 episodes were detected by both methods. Detection Method I uniquely identified 227 episodes, whereas Detection Method II uniquely identified 495 episodes (Supplementary Fig. 1 ).

The manual EHR review of the 800 patients from the three samples showed that 44 patients (5.5%) did not actually have MS (incorrect documentation). The most commonly misdiagnosed neurological diseases were neurosarcoidosis ( n = 3), myelin oligodendrocyte glycoprotein-associated disease (MOGAD) ( n = 3), and neuromyelitis optica spectrum disorder (NMOSD) ( n = 3), while in 12 patients (1.6%) the study neurologists were unable to determine relapse status due to missing clinical documentation.

The manual review of the sample created using Sampling Method I (i.e., 174 cases detected by either method and 226 controls) identified 87 false positives and two false negatives after neurologist validation (Supplementary Table 2 ). The review of the two samples created by Sampling Method II identified 46 false-positive cases and no false-negatives in the sample of cases detected by Detection Method I, and 48 false-positive and two false-negative cases in the sample of cases detected by Detection Method II (Supplementary Table 2 ).

False positives were most commonly attributed to unconfirmed relapses ( n = 61, 33.7%; e.g., corticosteroid use without supportive MRI or clinical evidence) and other neurological conditions ( n = 57, 31.5%; e.g., seizures or tumors), followed by cases with missing documentation ( n = 23, 12.7%; e.g., incomplete notes or incorrect records), pseudo-relapses ( n = 20, 11%; e.g., infection-related such as urinary tract infections and tuberculosis), and MS-related non-relapse admissions ( n = 20, 11%; e.g., infusion visits or disease complications) (Supplementary Fig. 2 ).

Table 3 shows the validation metrics for the application of the study’s detection methods in the three study samples. In the main study sample, 174 cases vs. 226 controls, sensitivity was 0.98 (95% CI, 0.92–0.99) and NPV was 0.99 (95% CI, 0.97–1.00), whereas specificity was 0.72 (95% CI, 0.67–0.77) and PPV was 0.50 (95% CI, 0.43–0.57). In the other two samples created by Sampling Method II, the application of Detection Method I showed high NPV (1.00, 95% CI 0.96–1.00) and moderate PPV (0.54, 95% CI 0.44–0.63). The application of Detection Method II yielded NPV and PPV that are almost comparable to those of Detection Method I: 0.98 (95% CI 0.93–0.99) and 0.52 (95% CI 0.42–0.62), respectively (Table 3 ).

Discussion

Our study demonstrated that both detection methods exhibited high sensitivity and NPV, whereas specificity and PPV were moderate. These findings suggest that the study detection methods perform well in detecting relapse episodes among hospitalized patients; however, additional confirmatory steps are required to ensure that the patients identified by these methods have truly experienced relapse episodes.

The relapse-detection methods in this study have higher sensitivity and NPVs compared with previously published methods [ 9 , 10 , 21 , 27 , 34 ]. Published studies have documented sensitivities and NPVs ranging from 70% to 75% and 70% to 96%; respectively [ 9 , 10 , 21 , 27 , 34 ]. However, specificity and PPVs of our methods are lower compared with those developed in the published literature, where specificities and PPVs ranged from 85% to 100% and 58.9% to 100%, respectively [ 9 , 10 , 21 , 27 ]. The moderate specificity and PPVs observed in our study may be attributed to the absence of structured documentation for key relapse-diagnostic variables, as well as several clinical factors [ 35 – 42 ]. First, the relatively low frequency of relapse episodes in the cohort inherently reduces PPV, as even highly sensitive detection methods yield a higher proportion of false positives when true events are rare, consistent with the known influence of disease prevalence on predictive values [ 35 , 36 ]. Second, high-dose corticosteroids can be prescribed for pseudo-relapses, transient symptoms worsening, or inflammatory conditions that fulfill the rules of the detection methods but do not represent true relapse episodes [ 37 , 38 ]. Additionally, institutional differences in hospitalization practices may affect how relapse episodes are captured in structured data [ 39 , 40 ]. Prior studies reporting higher specificity and PPV often incorporated additional variables such as plasmapheresis, or leveraged natural language processing (NLP) to extract key variables like EDSS, MSFC, and magnetic resonance imaging (MRI) findings from unstructured notes [ 21 , 41 ]. It is important to note that a significant proportion of healthcare institutions in Saudi Arabia, including this study’s center, continue to rely on unstructured notes to document details pertaining to disease diagnoses and treatments [ 42 ]. Enhancing the documentation of key MS diagnostic variables and/or employing tools such as NLP may contribute to improving the performance of existing and forthcoming relapse-detection approaches.

Given the high sensitivity and moderate PPV of our detection methods, they support the utility of structured EHR data in large-scale real-world research; however, we recommend applying refinement strategies which may include an initial chart review of a small random sample as a practical validation step, excluding admissions with concurrent infections, metabolic disturbances, or alternative neurologic diagnoses, and incorporating structured proxies including MRI orders, repeat corticosteroid administration, or MS treatment escalation following the event [ 38 , 43 ]. For studies intended to inform health authority decision-making, integration of unstructured data elements, NLP-based extraction of clinical severity markers, or targeted manual confirmation may be necessary to achieve the level of precision expected for regulatory standards.

Data for this study were obtained from a large healthcare institution spanning three regions in Saudi Arabia. However, assessing the applicability of these detection methods to other local or regional healthcare databases remains challenging due to the lack of published studies characterizing the nature and quality of local and regional MS RWD. As a result, the extent to which the study’s relapse detection methods can be generalized or applied in other local or regional contexts will need to be established through additional validation assessments. In settings where outpatient treatment or lower-dose corticosteroid regimens are more common, the current methods may preferentially identify more severe relapses requiring high-dose corticosteroids and hospitalization, while potentially underdetecting low- to moderate-severity relapses managed in outpatient settings. Future studies with larger sample sizes should evaluate alternative rule combinations, such as outpatient visits linked to corticosteroid dispensing, to assess their performance and potential to improve detection of milder relapse events.

This study is the first in the region to develop and validate EHR-based relapse-detection methods, directly supporting the conduct of real-world MS research using local data. The findings provide a foundation for pharmacoepidemiological studies on MS treatments in Saudi Arabia and the Middle East region, where such validated tools have been lacking. These methods were derived from structured EHR variables, specifically adapting corticosteroid dosing and hospitalization duration criteria, while aligning them with current treatment protocols to enhance reproducibility and applicability in settings with limited access to unstructured data. Different sampling methods were employed to enhance validity. Future research incorporating both structured and unstructured EHR data from multiple healthcare institutions for external validation and methods development would likely improve the accuracy of the current models and contribute to the creation of highly valid relapse-detection approaches.

Conclusion

We developed rule-based relapse-detection methods that demonstrate strong performance in identifying potential relapse episodes but also misclassify some non-relapse patients. While the high sensitivity indicates that these methods are effective in minimizing missed relapse events, the moderate PPV suggests that using structured EHR data alone may not currently achieve the precision required for effectiveness studies without confirmatory procedures. Therefore, additional confirmatory procedures are necessary to definitively ascertain patients experiencing relapse. Future multicenter studies, incorporating improved documentation of key MS diagnostic variables—particularly EDSS scores and MRI findings— as well as access to unstructured clinical notes, are essential for the external validation of the current models and the development of highly accurate relapse-detection approaches.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary Material 1

Acknowledgements

Not applicable.

Author contributions

Conceptualization: Turki Althunian, Adel Alrwisan, Ohoud Almadani, Almaha Alfakhri, Omar Albalawi, Yaser Al Malik, Ahmad Abulaban- Data Curation: Almaha Alfakhri, Ohoud Almadani, Yaser Al Malik, Ahmad Abulaban- Formal Analysis: Almaha Alfakhri, Ohoud Almadani, Turki Althunian- Methodology: Turki Althunian, Adel Alrwisan, Talal Alshihayb, Yasser Albogami- Validation: Shymaa Alkahtani, Renad Alkhalifah, Yaser Al Malik, Ahmad Abulaban- Writing Original Draft: Almaha Alfakhri, Ohoud Almadani- Writing Review & Editing: All authors- Supervision: Turki Althunian.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability

The data that support the study findings are not publicly available and cannot be shared due to confidentiality restrictions and institutional policies.

Declarations

Ethical approval

The study was conducted in accordance with the Declaration of Helsinki and has been approved by the institutional review board of Saudi Food and Drug Authority (SFDA) (Approval #2023_13) and King Abdullah International Medical Research Center (KAIMRC) (Approval #NRC23R/589/08); individual patient consent was waived by the approving ethics committees due to the retrospective design and use of de-identified data.

Consent for publication

Not applicable. All data used in this study were fully de-identified, and no information that could lead to the identification of individual participants is included in this manuscript.

Disclaimer

The views expressed in this paper are those of the author(s) and do not necessarily reflect those of the SFDA or its stakeholders. Guaranteeing the accuracy and the validity of the data is a sole responsibility of the research team.

Competing interests

The authors declare no competing interests.

Footnotes

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Cohen JA, Barkhof F, Comi G, Hartung HP, Khatri BO, Montalban X, et al. Oral fingolimod or intramuscular interferon for relapsing multiple sclerosis. N Engl J Med. 2010;362:402–15. 10.1056/NEJMoa0907839.

2. Cohen J, Belova A, Selmaj K, Wolf C, Sormani MP, Oberyé J, et al. Equivalence of Generic Glatiramer Acetate in Multiple Sclerosis. JAMA Neurol. 2015;72:1433. 10.1001/jamaneurol.2015.2154.

3. Hauser SL, Bar-Or A, Comi G, Giovannoni G, Hartung H-P, Hemmer B, et al. Ocrelizumab versus Interferon Beta-1a in Relapsing Multiple Sclerosis. N Engl J Med. 2017;376:221–34. 10.1056/NEJMoa1601277.

4. Fox RJ, Miller DH, Phillips JT, Hutchinson M, Havrdova E, Kita M, et al. Placebo-Controlled Phase 3 Study of Oral BG-12 or Glatiramer in Multiple Sclerosis. N Engl J Med. 2012;367:1087–97. 10.1056/NEJMoa1206328.

5. O’Connor P, Wolinsky JS, Confavreux C, Comi G, Kappos L, Olsson TP, et al. Randomized Trial of Oral Teriflunomide for Relapsing Multiple Sclerosis. N Engl J Med. 2011;365:1293–303. 10.1056/NEJMoa1014656.

6. Polman CH, O’Connor PW, Havrdova E, Hutchinson M, Kappos L, Miller DH, et al. A Randomized, Placebo-Controlled Trial of Natalizumab for Relapsing Multiple Sclerosis. N Engl J Med. 2006;354:899–910. 10.1056/NEJMoa044397.

7. Wang C, Ruiz A, Mao-Draayer Y. Assessment and treatment strategies for a multiple sclerosis relapse. J Immunol Clin Res. 2018;5.

8. Ross AP, Halper J, Harris CJ. Assessing relapses and response to relapse treatment in patients with multiple sclerosis: a nursing perspective. Int J MS Care. 2012;14:148–59. 10.7224/1537-2073-14.3.148.

9. Marriott JJ, Chen H, Fransoo R, Marrie RA. Validation of an algorithm to detect severe MS relapses in administrative health databases. Mult Scler Relat Disord. 2018;19:134–9. 10.1016/j.msard.2017.11.022.

10. Chastek BJ, Oleen-Burkey M, Lopez-Bresnahan MV. Medical chart validation of an algorithm for identifying multiple sclerosis relapse in healthcare claims. J Med Econ. 2010;13:618–25. 10.3111/13696998.2010.523670.

11. Liang L, Kim N, Hou J, Cai T, Dahal K, Lin C, et al. Temporal trends of multiple sclerosis disease activity: Electronic health records indicators. Mult Scler Relat Disord. 2022;57:103333. 10.1016/j.msard.2021.103333.

12. Flachenecker P, Buckow K, Pugliatti M, Kes VB, Battaglia MA, Boyko A, et al. Multiple sclerosis registries in Europe – results of a systematic survey. Multiple Scler J. 2014;20:1523–32. 10.1177/1352458514528760.

13. Cohen JA, Trojano M, Mowry EM, Uitdehaag BM, Reingold SC, Marrie RA. Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment. Mult Scler. 2020;26:23–37. 10.1177/1352458519892555.

14. Ohlmeier C, Gothe H, Haas J, Osowski U, Weinhold C, Blauwitz S, et al. Epidemiology, characteristics and treatment of patients with relapsing remitting multiple sclerosis and incidence of high disease activity: Real world evidence based on German claims data. PLoS ONE. 2020;15:e0231846–0231846. 10.1371/journal.pone.0231846.

15. Alsop J, Medin J, Cornelissen C, Vormfelde SV, Ziemssen T. Two studies in one: A propensity-score-matched comparison of fingolimod versus interferons and glatiramer acetate using real-world data from the independent German studies, PANGAEA and PEARL. PLoS ONE. 2017;12:e0173353–0173353. 10.1371/journal.pone.0173353.

16. Izquierdo G, Damas F, Paramo MD, Ruiz-Pena JL, Navarro G. The real-world effectiveness and safety of fingolimod in relapsing-remitting multiple sclerosis patients: An observational study. PLoS ONE. 2017;12:e0176174–0176174. 10.1371/journal.pone.0176174.

17. Yang CC, Ro LS, Tsai NW, Lin CC, Huang WN, Tsai CP, et al. Real-world evidence on the safety and effectiveness of fingolimod in patients with multiple sclerosis from Taiwan. J Formos Med Assoc. 2021;120(1 Pt 2):542–50. 10.1016/j.jfma.2020.07.002.

18. Batista S, Nunes CC, Cerqueira JJ, Martins Silva A, Correia de Sa J, Ferreira J, et al. REALMS study: real-world effectiveness and safety of fingolimod in patients with relapsing-remitting multiple sclerosis in Portugal. Neurol Sci. 2021;42:1995–2003. 10.1007/s10072-020-04726-6.

19. Bergvall N, Makin C, Lahoz R, Agashivala N, Pradhan A, Capkun G, et al. Relapse rates in patients with multiple sclerosis switching from interferon to fingolimod or glatiramer acetate: a US claims database study. PLoS ONE. 2014;9:e88472–88472. 10.1371/journal.pone.0088472.

20. Nicholas J, Boster A, Wu N, Yeh WS, Fay M, Kendter J, et al. Pharmacoecon open. 2018;2:31–41. 10.1007/s41669-017-0035-2. Comparison of Disease-Modifying Therapies for the Management of Multiple Sclerosis: Analysis of Healthcare Resource Utilization and Relapse Rates from US Insurance Claims Data.

21. Bosco-Levy P, Blin P, Lignot-Maleyran S, Lassalle R, Abouelfath A, Diez P, et al. Pnd75 Validation of an Algorithm Identifying Relapses in Multiple Sclerosis Using the French Nationwide Claims Database. Value Health. 2019;22:S750–1. 10.1016/j.jval.2019.09.1845.

22. Ezabadi SG, Sahraian MA, Maroufi H, Shahrbaf MA, Eskandarieh S. Global assessment of characteristics of multiple sclerosis registries; A systematic review. Mult Scler Relat Disord. 2022;63:103928. 10.1016/j.msard.2022.103928.

23. Knevel R, Liao KP. From real-world electronic health record data to real-world results using artificial intelligence. Ann Rheum Dis. 2023;82:306–11. 10.1136/ard-2022-222626.

24. Casey JA, Schwartz BS, Stewart WF, Adler NE. Using Electronic Health Records for Population Health Research: A Review of Methods and Applications. Annu Rev Public Health. 2016;37:61–81. 10.1146/annurev-publhealth-032315-021353.

25. Alnofal FA, Alrwisan AA, Alshammari TM. Real-world data in Saudi Arabia: Current situation and challenges for regulatory decision-making. Pharmacoepidemiol Drug Saf. 2020;29:1303–6. 10.1002/pds.5025.

26. Althunian TA, Alrasheed MM, Alnofal FA, Tafish RT, Mira MA, Alroba RA, et al. Recording type 2 diabetes mellitus in a standardised central Saudi database: a retrospective validation study. BMJ Open. 2023;13:e065468. 10.1136/bmjopen-2022-065468.

27. Colais P, Agabiti N, Davoli M, Buttari F, Centonze D, De Fino C, et al. Identifying Relapses in Multiple Sclerosis Patients through Administrative Data: A Validation Study in the Lazio Region, Italy. Neuroepidemiology. 2017;48:171–8. 10.1159/000479515.

28. Nicolaides NC, Pavlaki AN, Maria Alexandra MA, et al. Glucocorticoid Therapy and Adrenal Suppression. In: Feingold KR, Ahmed SF, Anawalt B, et al. editors. Endotext [Internet]. South Dartmouth (MA): MDText.com, Inc.; 2018.

29. Thurin NH, Bosco-Levy P, Blin P, Rouyer M, Jové J, Lamarque S, et al. Intra-database validation of case-identifying algorithms using reconstituted electronic health records from healthcare claims data. BMC Med Res Methodol. 2021;21:95. 10.1186/s12874-021-01285-y.

30. National Clinical Guideline Centre (UK). Multiple Sclerosis: Management of Multiple Sclerosis in Primary and Secondary Care. London, UK: National Institute for Health and Care Excellence; 2014.

31. Fox MP, Lash TL, Bodnar LM. Common misconceptions about validation studies. Int J Epidemiol. 2020;49:1392–6. 10.1093/ije/dyaa090.

32. Trevethan R, Sensitivity, Specificity, Values P. Foundations, Pliabilities, and Pitfalls in Research and Practice. Front Public Health. 2017;5:307. 10.3389/fpubh.2017.00307.

33. Monaghan TF, Rahman SN, Agudelo CW, Wein AJ, Lazar JM, Everaert K, et al. Foundational statistical principles in medical research: sensitivity, specificity, positive predictive value, and negative predictive value. Med (Kaunas). 2021;57. 10.3390/medicina57050503.

34. Carcaillon-Bentata L, Abouelfath A, Lignot-Maleyran S, Lassalle R, Jové J, Brochet B et al. Validation and improvement of an algorithm identifying relapses in multiple sclerosis (MS) within the French nationwide claims database. In: ISPE Annual Meeting. Washington, DC, USA: International Society for Pharmacoepidemiology; 2025.

35. Grimes DA, Schulz KF. Uses and abuses of screening tests. Lancet. 2002;359:881–4. 10.1016/S0140-6736(02)07948-5.

36. Maxim LD, Niebo R, Utell MJ. Screening tests: a review with examples. Inhal Toxicol. 2014;26:811–28. 10.3109/08958378.2014.955932.

37. Kalincik T, Manouchehrinia A, Sobisek L, Jokubaitis V, Spelman T, Horakova D, et al. Towards personalized therapy for multiple sclerosis: prediction of individual treatment response. Brain. 2017;140:2426–43. 10.1093/brain/awx185.

38. Salter A, Lancia S, Sharma M, Cutter GR, Fox RJ, Marrie RA. Infection, relapses, and pseudo-relapses in individuals with multiple sclerosis. Neurol Clin Pract. 2025;15. 10.1212/CPJ.0000000000200493.

39. Paredes D, Lackey E, Shah S. Reasons for Hospital Admission in Individuals With Multiple Sclerosis. Int J MS Care. 2024;26:302–7. 10.7224/1537-2073.2023-064.

40. Cabreira V, Abreu P, Maia C, Costa A, Sá MJ. Trends in hospital readmissions in Multiple Sclerosis patients between 2009 and 2015. Mult Scler Relat Disord. 2020;45:102396. 10.1016/j.msard.2020.102396.

41. Chin I, Moss H, Sands K, Kocher M, Akkas O, Torres A. Identifying Multiple Sclerosis Relapses from Clinical Notes Using Combined Rule-based and Deep Learning Methodologies (P3-3.001). Neurology. 2023;100:17supplement22829. 10.1212/wnl.0000000000202826.

42. Alharthi H. Healthcare predictive analytics: An overview with a focus on Saudi Arabia. J Infect Public Health. 2018;11:749–56. 10.1016/j.jiph.2018.02.005.

43. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, et al. Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria. Ann Neurol. 2011;69:292–302. 10.1002/ana.22366.

Associated Data

Data Citations

Nicholas J, Boster A, Wu N, Yeh WS, Fay M, Kendter J, et al. Pharmacoecon open. 2018;2:31–41. 10.1007/s41669-017-0035-2. Comparison of Disease-Modifying Therapies for the Management of Multiple Sclerosis: Analysis of Healthcare Resource Utilization and Relapse Rates from US Insurance Claims Data.

Supplementary Materials

Supplementary Material 1

Data Availability Statement

The data that support the study findings are not publicly available and cannot be shared due to confidentiality restrictions and institutional policies.