The next best teacher: Using large language models to provide technical advice to trainees in regional Anaesthesia.

BACKGROUND

We assessed three Large Language Models (LLMs) ability to give safe and accurate information to anaesthetic trainees for multiple different regional anaesthesia techniques.

METHODS

This study was a single-centre comparative cross-sectional study of three common LLMs regarding their advice on seven regional nerve blocks. The large language models evaluated included ChatGPT (OpenAI, USA), Bing Chat/Copilot (Microsoft Corporation, USA), and Gemini (Google DeepMind, Google LLC, USA). The advice given by the LLM was then appraised by ten anaesthetists for safety, accuracy and appropriateness for trainees. The LLMs were then compared between nerve blocks. The readability of the advice was analysed to evaluate the output's complexity. The references provided by the LLMs were evaluated to ensure their accuracy.

RESULTS

ChatGPT had a mean score of 3.71 out of 5 across all seven regional blocks and an overall Cronbach Alpha of 0.98 (95% CI 0.89-0.99). ChatGPT scored higher than Gemini for all nerve blocks investigated and higher than Bing for femoral nerve blocks. The readability of the outputs from ChatGPT for both the Flesch Reading Ease Score and Flesch-Kincaid Grade level indicated that the advice was intended to be read by a University Graduate. All the references supplied by Bing and Gemini were real, whereas only 25% of the references supplied by ChatGPT existed.

CONCLUSIONS

LLM have a role to play in intraprocedural queries, however, should only be used as an adjunct to the decision-making process and need to be critically appraised. The utility of LLM in medical education and intraprocedural queries will need ongoing evaluation due to the technology's continuing evolution.

REWORDED KEY MESSAGE: There were no studies using LLMs as an educational tool in regional anaesthesia, the aim was to validate if LLMs were an option for trainee anaesthetists. This study shows that LLMs output can give valuable insights to trainees, although need to be critically analysed before being used to ensure patient safety. Further comparative studies versing other teaching modalities need to be conducted to see if LLMs can provide the best and safest information to trainees practicing regional anaesthesia.