MedElevator Войти в исследования
← Поиск публикаций

DOI 10.1109/tnnls.2026.3726274

Медицинская библиотека · IEEE transactions on neural networks and learning systems · 2026

Focus Where It Counts: A Salience-Driven Vision-Language Model for Low Vision Assistance.

Liang Jiazhao, Huang Hao, Yuan Shuaihang, Wen Congcong, Bethala Geeta Chandra Raju, Hamilton-Fletcher Giles, Hao Yu, Rizzo John-Ross, Wang Mengyu, Tzes Anthony, Fang Yi

СтатьяПеревод в очереди

Аннотация

Vision-language models (VLMs) are rapidly progressing and offer promising capabilities for assistive technologies supporting persons with blindness or low vision. However, existing VLMs are primarily designed for general-purpose captioning and do not explicitly model human perceptual priorities, thereby limiting their ability to emphasize the most relevant information in a scene. To address this gap, we propose a salience-driven captioning framework that prioritizes scene elements according to their importance for human-centered assistance. We curate three salience-aware datasets, namely, Salience COCO, Salience Flickr, and Salience Vizwiz, with object-level salience annotations designed to reflect the visual information most relevant to low vision users across different environments. Building on these datasets, we introduce Salience-LLaVA, a salience-aware VLM that incorporates salience cues to generate captions in which important elements are mentioned in the order of importance. Our work makes four main contributions. We build salience-aware datasets verified by low vision participants, propose Salience-LLaVA to describe objects in the order of importance, introduce SCMI to evaluate ordering accuracy, and deploy the system on assistive glasses to demonstrate its real-world practicality. Code and datasets are available at: https://github.com/topo-focus/Topofocus.

Для цитирования: Liang Jiazhao, Huang Hao, Yuan Shuaihang, Wen Congcong, Bethala Geeta Chandra Raju, Hamilton-Fletcher Giles, Hao Yu, Rizzo John-Ross, Wang Mengyu, Tzes Anthony, Fang Yi. Focus Where It Counts: A Salience-Driven Vision-Language Model for Low Vision Assistance.. IEEE transactions on neural networks and learning systems. 2026. DOI: 10.1109/tnnls.2026.3726274.
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…

История · было → стало

Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.

Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.