Clinician expertise and prompt engineering enhance cancer information extraction in electronic health records by small language models.
Обработка статьи · этапы, ошибки и управление
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Corso Federica, Peppoloni Vittoria, Mazzeo Laura, Leone Giuseppe, Passos Luana, Mišković Vanja, Armanini Justin, Ferrarin Alberto, Wiest Isabella Catharina, Wolf Fabian, Montelatici Giulia, Romanò Rebecca, Ambrosini Paolo, Capoccia Tommaso, Natangelo Stefano, Rota Simone, Andena Paola, De Ponti Marta, Russo Alessandra, Stasi Giulia, Provenzano Leonardo, Spagnoletti Andrea, Meazza Prina Marco, Cavalli Chiara, Giani Claudia, Serino Roberta, Borracino Michele, Bonalume Chiara, Di Mauro Rosa Maria, Agosta Claudia, Dumitrascu Andra Diana, Di Liberti Giorgia, Corrao Giulia, Beninato Teresa, Ganzinelli Monica, Occhipinti Mario, Brambilla Marta, Proto Claudia, Kather Jakob Nikolas, Pedrocchi Alessandra Laura Giulia, De Braud Filippo, Lo Russo Giuseppe, Baili Paolo, Prelaj Arsela
Аннотация
METHODS: We examine three prompting strategies (zero-shot, few-shot, and annotated few-shot) across English and Italian, involving clinicians with varying expertise to assess the impact of prompt design on accuracy. We evaluate the performance of four open-source small language models (SLMs) for clinical information extraction from Italian electronic health records (EHRs) in the APOLLO 11 trial on non-small cell lung cancer (NSCLC). The extraction protocol involves four steps: problem definition, data preprocessing, Large Language Model (LLM)-based information extraction, and output evaluation.
RESULTS: We show that general-purpose models (e.g., LLaMA 3.1 8B) outperform biomedical models in most tasks, particularly in extracting binary features. Multiclass variables such as TNM (Tumor, Node, Metastasis) staging, PD-L1 (Programmed death-ligand 1), and ECOG-PS (Eastern Cooperative Oncology Group-Performance Status) are more difficult due to implicit language and lack of standardization. Few-shot prompting and native-language inputs significantly improve performance and reduced hallucinations. Clinical expertise enhances consistency in the extraction, particularly among students using annotated examples.
CONCLUSIONS: The study confirms that privacy-preserving SLMs can be deployed locally for efficient and secure cancer data extraction. Findings highlight the need for hybrid systems combining SLMs with expert input and underline the importance of aligning clinical documentation practices with SLM capabilities. This is the first study to benchmark SLMs on Italian EHRs and investigate the role of clinical expertise in prompt engineering, offering valuable insights for the future integration of SLMs into real-world clinical workflows.