Using conversational signals for acute care risk stratification: A multimodal AI approach in home health care.

BACKGROUND

Acute care utilization remains common during home health care episodes.

OBJECTIVE

This study evaluated whether conversational data from follow-up phone calls can predict hospitalization or emergency department utilization within 30 and 60 days.

DESIGN

Prospective observational cohort study.

SETTING

A large not-for-profit home health care agency in the United States.

PARTICIPANTS

A total of 174 adult patients receiving home health care who could communicate independently in English and completed at least one follow-up telephone call.

METHODS

Participants completed up to four weekly, structured follow-up calls conducted by a trained research assistant during a single home health care episode. Using a multimodal framework, we developed single-representation models based on: (1) transcript-derived features (large language models and ClinicalBERT), (2) audio-derived features (engineered acoustic features and wav2vec2 embeddings), and (3) one-minute audio segments analyzed with audio-enabled language models. Multimodal risk was estimated by combining predictions across representations. Performance was assessed using five-fold grouped cross-validation with out-of-fold predictions. The primary outcome was hospitalization or emergency department utilization within 30 days of the first call; the secondary outcome was within 60 days. Clinical utility was evaluated via risk stratification (low, moderate, high) and logistic regression.

RESULTS

Hospitalization or emergency department utilization occurred in 10.9% of patients within 30 days and 18.4% within 60 days. The best-performing single-representation models achieved areas under the receiver operating characteristic curve of 0.70 and 0.71 for transcript-based approaches, 0.77 and 0.74 for audio-derived feature approaches, and 0.69 and 0.70 for one-minute audio-segment approaches at 30 and 60 days, respectively. Multimodal integration improved discrimination (area under the receiver operating characteristic curve: 0.85 within 30 days and 0.79 within 60 days). Risk stratification showed a clear gradient: compared with low-risk patients, high-risk patients had 44.65-fold higher odds within 30 days (95% confidence interval: 2.61-765.15) and 13.31-fold higher odds within 60 days (95% confidence interval: 3.37-52.61). Predictive gains persisted using only the first call.

CONCLUSIONS

Brief follow-up calls contain clinically meaningful signals for predicting acute care use in home health care. Multimodal integration enables scalable, actionable risk stratification to support proactive care planning.

REGISTRATION: Not registered.