Pakistan journal of medical sciences · 2026
Beyond Red Pen: Comparing the machine vs human grading of reflective assignments on clinical reasoning in Dermatology undergraduate students.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Faysal Lubna Rani, Ali Shabana, Aftab Khan Mahwish, Fayyaz Khan Humaira
Аннотация
BACKGROUND AND OBJECTIVE: The integration of Artificial Intelligence (AI) into medical education has been a significant advancement in recent years. AI-based tools like ChatGPT offer numerous advantages for teaching and student assessment. The objective of this study was to evaluate the effectiveness of ChatGPT in assessing reflective assignments compared to human grading.
METHODOLOGY: This quasi-experimental study was conducted at Islamic International Medical College from December 2024 to March 2025. In this study a total of 120 3rd year MBBS students were assigned reflective essays after completing their Dermatology module. The e-assignment focused on reflecting on diagnosing scabies, applying clinical reasoning, and correlating clinical findings with patient history, following the Gibbs Reflective Cycle. Each submission was standardized to 500 words and graded using a structured rubric. First the selected faculty members manually assessed each assignment, and then ChatGPT applied the same rubric for grading of the assignments. The scores were then compared for alignment between human and ChatGPT grading.
RESULTS: It showed a strong correlation between ChatGPT and human scores, with minimal differences of 0.5-1.5 marks in a few cases. Clinical Reasoning scores showed a strong correlation (ρ = 0.678, p < 7×10-9) with a small effect size (d = -0.283), while Gibbs Cycle scores had an even stronger correlation (ρ = 0.734, p < 1×10-10) and negligible effect size (d = 0.037). Total scores showed very high correlation (ρ = 0.990, p < 1×10-10) but a large effect size (d = 3.422), suggesting consistent scoring with systematic differences in absolute values.
CONCLUSION: ChatGPT serves as a valuable and reliable complement to human assessment, significantly improving grading efficiency, particularly in large-scale evaluations.
METHODOLOGY: This quasi-experimental study was conducted at Islamic International Medical College from December 2024 to March 2025. In this study a total of 120 3rd year MBBS students were assigned reflective essays after completing their Dermatology module. The e-assignment focused on reflecting on diagnosing scabies, applying clinical reasoning, and correlating clinical findings with patient history, following the Gibbs Reflective Cycle. Each submission was standardized to 500 words and graded using a structured rubric. First the selected faculty members manually assessed each assignment, and then ChatGPT applied the same rubric for grading of the assignments. The scores were then compared for alignment between human and ChatGPT grading.
RESULTS: It showed a strong correlation between ChatGPT and human scores, with minimal differences of 0.5-1.5 marks in a few cases. Clinical Reasoning scores showed a strong correlation (ρ = 0.678, p < 7×10-9) with a small effect size (d = -0.283), while Gibbs Cycle scores had an even stronger correlation (ρ = 0.734, p < 1×10-10) and negligible effect size (d = 0.037). Total scores showed very high correlation (ρ = 0.990, p < 1×10-10) but a large effect size (d = 3.422), suggesting consistent scoring with systematic differences in absolute values.
CONCLUSION: ChatGPT serves as a valuable and reliable complement to human assessment, significantly improving grading efficiency, particularly in large-scale evaluations.