Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials.

Artificial intelligence (AI) is increasingly used in healthcare to identify high-risk patients. Laboratory studies are mixed regarding patient attitudes towards AI use. Here we field-tested whether informing patients of their risk for influenza and its complications can serve as an effective behaviour-change intervention to increase vaccination, and whether disclosing AI use in the risk determination or providing personalized reasons for the algorithm's prediction (a form of explainable AI) influence message effectiveness. We ran three preregistered randomized controlled trials with over 90,000 unique healthcare system patients in the USA (ClinicalTrials.gov identifiers: NCT04323137 , NCT05009251 and NCT05509283 ). Patients identified by a previously validated machine learning algorithm as being at high risk for influenza and related complications were randomized to be sent no message or one of several different messages encouraging influenza vaccination (the primary outcome). High-risk nudges increased vaccination: among patients informed of their high risk, vaccination was 1.1-1.4 percentage points (3.3-5.4%) higher than those simply reminded to get a vaccine, and 1.7-3.5 percentage points (3.3-14.7%) higher than non-messaged patients. Vaccination was similar across message arms that did versus did not disclose 'algorithm' involvement or its risk explanations, indicating that patients are neither averse to nor appreciative of algorithm use or explainability in this realistic health application. This work was partially funded by the National Institute on Aging of the National Institutes of Health under award number P30AG034532.