Diabetes research and clinical practice · 2026
Evaluation of large language models for carbohydrate counting in type 1 diabetes mellitus: a comparative study with patient estimations.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Soyer Ahmet Kürşat, Akarsu Gamze Çelen, Yelli Belgin Demir, Tural Balsak Belma Özlem, Özdemir Didem, Topaloğlu Oya, Ersoy Reyhan, Çakır Bekir
Аннотация
AIMS: To evaluate large language models (LLMs) accuracy in carbohydrate (CHO) counting (CC) and compare their performance with estimations by patients with type-1 diabetes mellitus (T1DM).
METHODS: This cross-sectional study included 14 adults with T1DM with advanced CC training and > 70% accuracy on the AdultCarbQuiz. Eighteen main meals and four snacks were prepared by dietitians using standardized descriptions and two-angle photographs. Dietitian-determined values served as the reference. CHO estimations by patients and LLMs (ChatGPT-5.2, Gemini-3 Pro, and Claude Sonnet-4.5) were compared. LLM performance was evaluated in two-stages: image-only (stage-1) and image-plus-detailed-description (stage-2) inputs. Performance was assessed using mean absolute error (MAE), mean absolute percentage error (MAPE), counting accuracy (CA; ≤10 g), Wilcoxon signed-rank test, and Bland-Altman analysis.
RESULTS: Patients showed MAE 8.31 g, MAPE 13.82%, and CA 73.74%. In Stage-1, Gemini had the lowest MAE (6.55 g), and MAPE (12.54%), while Claude achieved the highest CA (81.8%). In Stage-2, Gemini outperformed patients (p < 0.001) with MAE 2.14 g, MAPE 3.21%, CA 100%, and mean bias - 1.05 g. Other model-stage combinations showed performance comparable to patients.
CONCLUSION: LLMs, particularly Gemini-3 Pro, may support CC when detailed inputs are provided; however, findings should be interpreted cautiously given the small, highly selected sample.
METHODS: This cross-sectional study included 14 adults with T1DM with advanced CC training and > 70% accuracy on the AdultCarbQuiz. Eighteen main meals and four snacks were prepared by dietitians using standardized descriptions and two-angle photographs. Dietitian-determined values served as the reference. CHO estimations by patients and LLMs (ChatGPT-5.2, Gemini-3 Pro, and Claude Sonnet-4.5) were compared. LLM performance was evaluated in two-stages: image-only (stage-1) and image-plus-detailed-description (stage-2) inputs. Performance was assessed using mean absolute error (MAE), mean absolute percentage error (MAPE), counting accuracy (CA; ≤10 g), Wilcoxon signed-rank test, and Bland-Altman analysis.
RESULTS: Patients showed MAE 8.31 g, MAPE 13.82%, and CA 73.74%. In Stage-1, Gemini had the lowest MAE (6.55 g), and MAPE (12.54%), while Claude achieved the highest CA (81.8%). In Stage-2, Gemini outperformed patients (p < 0.001) with MAE 2.14 g, MAPE 3.21%, CA 100%, and mean bias - 1.05 g. Other model-stage combinations showed performance comparable to patients.
CONCLUSION: LLMs, particularly Gemini-3 Pro, may support CC when detailed inputs are provided; however, findings should be interpreted cautiously given the small, highly selected sample.