Integration of natural language models in the diagnosis of systemic autoimmune diseases: validation of GPT-4 in a tertiary care center.

INTRODUCTION

Systemic autoimmune diseases (SADs) pose a diagnostic challenge due to the heterogeneity of their manifestations and the frequent overlap of symptoms. The integration of large language models (LLMs), such as GPT-4, could complement clinical judgment through the systematic analysis of standardized clinical data.

OBJECTIVE

To evaluate the diagnostic capability of GPT-4 in patients with SADs at a tertiary care center, comparing its results with the final consensus diagnosis issued by specialists.

METHODS

A retrospective study was conducted on a cohort of 101 consecutively treated patients between January 1 and March 31, 2024, at the SAD Unit of La Paz University Hospital. Data collection was carried out using the unit's standardized anamnesis protocol. The "my GPT" model, based on GPT-4 and trained according to international diagnostic criteria, was evaluated following TRIPOD-AI guidelines.

RESULTS

The overall diagnostic accuracy rate was 97.03%. Analysis based solely on anamnesis data achieved an accuracy of 82.18%, which increased by 14.85% when immunological results were included. A 100% accuracy was achieved in diagnosing systemic lupus erythematosus, Sjögren's syndrome, inflammatory myopathies, Behçet's disease, and scleroderma. In contrast, for sarcoidosis and vasculitis-conditions that often require histological confirmation-accuracy was 91.67% and 80%, respectively.

CONCLUSION

The use of GPT-4, grounded in systematic clinical data collection and evaluated in accordance with TRIPOD-AI guidelines, demonstrates strong potential as an auxiliary tool in the diagnosis of SADs. Integrating this approach into clinical practice could help reduce interobserver variability and optimize decision-making.