Enhancing Cancer Outcomes Research Through Vision Language Models for Cause of Death Ascertainment.

BACKGROUND AND SCOPE: Cause of death is an important variable in cancer registries that supports downstream analyses, including cause-specific survival analysis. Unfortunately, in current practice, it is only available on government-issued death certificates, which are often received as scanned images. Current computer vision-based optical character recognition (OCR) methods struggle with handwriting, background noise, and blurriness, resulting in poor performance. A robust method for the cause of death ascertainment is urgently needed.

SOLUTION: Latest vision language models (VLMs), such as Qwen3-VL and GPT-4.1, have shown promising OCR performance on poor-quality scanned documents in the general domain. We proposed two prompting methods: (1) the two-step approach, which prompts VLMs for a full-text OCR followed by a text-based information extraction and (2) end-to-end approach, which prompts VLMs with a JavaScript Object Notation schema for image-based information extraction.

EVALUATION: We evaluated VLMs for the cause of death extraction from scanned government-issued death certificates in a major cancer center. Of 22,733 death records, 500 were manually annotated as ground truth. For the immediate cause of death, Qwen3-VL-30B-A3B-Thinking with the end-to-end approach achieved the best accuracy of 0.8625, significantly outperforming the Tesseract baseline (0.2396). For the underlying causes of death, the two-step method achieved the best F1 score of 0.7637, outperforming the baseline (0.2451).

RELEVANCE: The growing emphasis on precision medicine underscores the importance of accurate cause of death ascertainment for survival analyses. Our method demonstrated significant improvements over the current OCR approaches and provided a reliable alternative to reliance on the National Death Index.

HOW TO USE: Our VLM OCR computational framework is publicly available on GitHub. The open-weight VLMs are available for download on the Hugging Face repository.