BJU international · 2025
Quality of patient information on interstitial cystitis from artificial intelligence chatbots.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Santucci Jordan, Stapleton Peter, Ibrahim Joe, Johns-Putra Lydia, Elmer Sandra, Sathianathen Niranjan
Аннотация
OBJECTIVE: To evaluate the quality (DISCERN), understandability and actionability (Patient Education Materials Assessment Tool for Printable Materials [PEMAT-P]), readability (Flesch-Kincaid), and misinformation of patient-facing information on interstitial cystitis generated by four publicly available artificial intelligence (AI) chatbots: ChatGPT-4.0, Perplexity, ChatSonic, and Bing AI.
METHODS: A total of 10 queries derived from Google Trends and Hopkins Medicine content were submitted to each chatbot. Responses were evaluated by two blinded reviewers using validated tools: the DISCERN instrument (reliability/quality), PEMAT-P (understandability/actionability), and Flesch-Kincaid Grade Level (readability). Word count and citation inclusion were also recorded.
RESULTS: Across chatbots, information quality was moderate with a median (interquartile range [IQR]) DISCERN score of 3/5 (2-3), with Perplexity performing best and Bing AI worst. Understandability was moderate (median [IQR] PEMAT-P score 75% [66.7-83.3%]), highest for ChatSonic with Hopkins Medicine-derived prompts and lowest for ChatGPT with Google Trends inputs. Actionability was consistently poor (median [IQR] score 40% [20-60%]), with ChatSonic performing best and Bing AI lowest. Responses averaged 256 words and college-level readability (median [IQR] Flesch-Kincaid score 25.4 [20.89-28.50]) across all platforms, limiting accessibility. Misinformation was minimal across all platforms. Chatbots referencing clinically curated prompts (Hopkins Medicine) scored higher in understandability and completeness than those responding to public search trends.
CONCLUSION: Artificial intelligence chatbots offer generally accurate and understandable information about interstitial cystitis but lack actionable guidance and generate content at reading levels above typical patient comprehension. Enhancing readability, actionability, and personalisation may increase their utility as adjunct tools for patient education in functional urology.
METHODS: A total of 10 queries derived from Google Trends and Hopkins Medicine content were submitted to each chatbot. Responses were evaluated by two blinded reviewers using validated tools: the DISCERN instrument (reliability/quality), PEMAT-P (understandability/actionability), and Flesch-Kincaid Grade Level (readability). Word count and citation inclusion were also recorded.
RESULTS: Across chatbots, information quality was moderate with a median (interquartile range [IQR]) DISCERN score of 3/5 (2-3), with Perplexity performing best and Bing AI worst. Understandability was moderate (median [IQR] PEMAT-P score 75% [66.7-83.3%]), highest for ChatSonic with Hopkins Medicine-derived prompts and lowest for ChatGPT with Google Trends inputs. Actionability was consistently poor (median [IQR] score 40% [20-60%]), with ChatSonic performing best and Bing AI lowest. Responses averaged 256 words and college-level readability (median [IQR] Flesch-Kincaid score 25.4 [20.89-28.50]) across all platforms, limiting accessibility. Misinformation was minimal across all platforms. Chatbots referencing clinically curated prompts (Hopkins Medicine) scored higher in understandability and completeness than those responding to public search trends.
CONCLUSION: Artificial intelligence chatbots offer generally accurate and understandable information about interstitial cystitis but lack actionable guidance and generate content at reading levels above typical patient comprehension. Enhancing readability, actionability, and personalisation may increase their utility as adjunct tools for patient education in functional urology.