Large language model responses to questions about brain death/death by neurologic criteria: ChatGPT 4o-mini and Gemini 3 Flash (Fast).

BACKGROUND

Public understanding about brain death/death by neurologic criteria (BD/DNC) is generally poor. With the rising popularity of utilization of large language model (LLM) chatbots to answer medical questions, we sought to determine the quality of information about BD/DNC provided by ChatGPT 4o-mini and Gemini 3 Flash (Fast).

METHODS

With the assistance of a family advocate, we developed 45 open-ended questions about BD/DNC and submitted them to ChatGPT 4o-mini and Gemini 3 Flash (Fast) in January 2026. We recorded response word count, Flesch-Kincaid Readability Score and source reputability (non-reputable sources were defined as nonmedical, nongovernmental, nonlegal and not affiliated with an organ donation organization). Two authors of the 2023 BD/DNC guidelines independently assessed response accuracy relative to accepted medical standards and a third adjudicated discrepancies.

RESULTS

Most responses were ≥ 10th grade level [ChatGPT 4o-mini: 44/45 (98%), Gemini 3 Flash (Fast): 39/45 (86%)]. After adjudication, 22/45 (49%) responses from Gemini 3 Flash (Fast) and 20/45 (44%) from ChatGPT 4o-mini were considered completely correct (p = 0.673). There was no relationship between accuracy and: word count; readability; or citation of at least one non-reputable source.

CONCLUSION

ChatGPT 4o-mini and Gemini 3 Flash (Fast) responses to questions about BD/DNC may include inaccuracies. This could promote confusion and distrust. There is remarkable potential for integration of artificial intelligence in public education about healthcare, but there is a need for improvement to ensure responses are accurate and readable. The healthcare team should be prepared to address misconceptions about BD/DNC based on use of LLMs.