A pragmatic risk-stratified framework for using large language models in intensive care medicine: A narrative review.

OBJECTIVE

To provide Australian intensive care clinicians with a pragmatic framework for the safe integration of large language models (LLMs) into intensive care unit (ICU) practise, addressing the current lack of Australian-specific guidance and limited local evidence.

DESIGN

Narrative review.

DATA SOURCES: Peer-reviewed publications, preprints, and policy documents relating to LLM use in health care, with a focus on critical care applications and governance.

REVIEW

METHODS

Evidence and expert commentary were synthesised to develop a clinician-led, risk-stratified framework for ICU implementation, with emphasis on safety, oversight, and applicability within Australian health systems. Clinical use cases, risks, governance considerations, and practical safeguards for day-to-day ICU practise were identified.

RESULTS

LLMs have potential utility in data-dense ICU environments, including summarising complex clinical information, supporting documentation, assisting clinical reasoning, and facilitating research tasks. However, evidence for LLM performance in ICU contexts remains limited, particularly in Australia. Key risks include inaccurate or fabricated outputs ("hallucinations"), bias, lack of situational awareness, privacy concerns, and over-reliance in high-stakes decision-making. We propose a risk-stratified framework that categorises LLM applications by clinical risk and reversibility, and aligns each category with proportional oversight, verification processes, and governance safeguards, emphasising clinician-in-the-loop decision-making.

CONCLUSIONS

LLMs may serve as adjunctive cognitive tools in Australian ICUs when used in clearly defined, low-to intermediate-risk contexts under clinician oversight. Safe integration requires robust governance frameworks emphasising transparency, data protection, and proportionate clinician decision-making. Further Australian-based evaluation is needed before high-risk clinical applications can be considered for routine practise.