Curriculum-Aligned Conversational AI to Support Pathology Self-Directed Learning in Preclinical Medical Students: A Crossover Randomized Pilot Study.

BACKGROUND

Artificial intelligence (AI) is reshaping the landscape of medical education, yet the widespread adoption of general-purpose AI tools in student learning raises substantial concerns about curricular alignment, information accuracy, and the absence of pedagogical scaffolding.

OBJECTIVE

This crossover pilot study evaluated a curriculum-aligned custom GPT that was developed to support AI-scaffolded self-directed learning (SDL), preparation that remains student-initiated but is supported by structured Socratic prompting, compared with fully unguided traditional preparation in second-year medical students at the University of South Carolina, Floyd School of Medicine (USC Floyd SoM).

METHODS

Forty-four second-year medical students voluntarily participated in the two faculty-led pathology small group discussions (Cushing's disease and prostate cancer), using the custom GPT for one case and conventional resources for the other. Faculty assessed SDL documentation forms and small group discussion performance using standardized rubrics.

RESULTS

Students using the curriculum-aligned custom GPT achieved significantly higher SDL rubric scores (10.3 vs. 9.4 out of 12; mean difference 0.9, 95% CI 0.5-1.3; P<.001; Cohen d=0.71) and small group discussion scores (14.9 vs. 13.7 out of 16; mean difference 1.2, 95% CI 0.7-1.7; P<.001; Cohen d=0.78), with moderate to large effect sizes. Delayed retention (end-of-block examination items) did not differ significantly (Wilcoxon signed-rank P=.052). Students also reported significantly greater gains in understanding and pathophysiological reasoning. 34 out of 44 (77%) reported improved reasoning clarity, and 31 out of 44 (70%) improved conceptual understanding. Assessment of five prespecified AI-related concern domains did not identify major concerns.

CONCLUSIONS

In this crossover pilot study, use of curriculum-aligned custom GPT was associated with higher faculty-assessed SDL documentation and small group discussion performance. Delayed knowledge retention did not differ significantly in the primary analysis. These findings support further investigation of curriculum-aligned AI tools in larger, multisite studies.

CLINICALTRIAL: This pilot study was not registered in a public trial registry. As a minimal-risk, exempt educational pilot study conducted within a single course block, formal registration was not pursued; the authors note this as a limitation and recommend prospective registration for future, larger-scale studies of this intervention.