International journal of computer assisted radiology and surgery · 2026
Real-time marker-less needle tracking for CT-guided interventions using multiple RGB cameras.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Steiger Max, Mielke Tonia, Bashkanov Oleksii, Grosser Oliver S, Pech Maciej, Hansen Christian, Rak Marko
Аннотация
PURPOSE: In CT-guided interventions, instrument tracking reduces patient risk while improving the overall outcome. Though useful, marker-based tracking systems suffer from occlusion, are complex to set up, and fail when needles deflect. A marker-less tracking system could overcome these challenges, enabling streamlined integration.
METHODS: We developed and evaluated a proof-of-concept system combining standard RGB cameras with a U-Net-ConvNeXt architecture for needle detection and tracking. We compiled a fully annotated dataset of 35,000 frames, spanning multiple needle types and scenarios. We evaluated tracking accuracy against state-of-the-art marker-based systems, utilizing an IR-based and an image-based system on static positioning, dynamic movement, and needle-deflection scenarios. Moreover, our marker-less system was evaluated using both known and unknown needle types.
RESULTS: Our proof-of-concept system demonstrated a 3D tracking error of 2.70 ± 1.07 mm at 25 FPS with 3 cameras, outperforming ArUco marker-based tracking ( 5.29 ± 1.85 mm). Tracking accuracy remained consistent for unknown needle types not included in the training data, suggesting strong generalization. The system maintained a 99.4 % needle-tip detection rate across all frames with robust performance during needle deflection, while marker-based systems failed under identical conditions. With only 2 cameras, the tracking error of our system increased modestly ( 4.81 ± 2.38 mm).
CONCLUSION: Our proof-of-concept demonstrates the feasibility of marker-less needle tracking, suggesting potential for clinically relevant accuracy, reliability, and reduced complexity while exhibiting robustness to needle deflection and varying needle appearances. These preliminary results indicate promising directions for future clinical development.
METHODS: We developed and evaluated a proof-of-concept system combining standard RGB cameras with a U-Net-ConvNeXt architecture for needle detection and tracking. We compiled a fully annotated dataset of 35,000 frames, spanning multiple needle types and scenarios. We evaluated tracking accuracy against state-of-the-art marker-based systems, utilizing an IR-based and an image-based system on static positioning, dynamic movement, and needle-deflection scenarios. Moreover, our marker-less system was evaluated using both known and unknown needle types.
RESULTS: Our proof-of-concept system demonstrated a 3D tracking error of 2.70 ± 1.07 mm at 25 FPS with 3 cameras, outperforming ArUco marker-based tracking ( 5.29 ± 1.85 mm). Tracking accuracy remained consistent for unknown needle types not included in the training data, suggesting strong generalization. The system maintained a 99.4 % needle-tip detection rate across all frames with robust performance during needle deflection, while marker-based systems failed under identical conditions. With only 2 cameras, the tracking error of our system increased modestly ( 4.81 ± 2.38 mm).
CONCLUSION: Our proof-of-concept demonstrates the feasibility of marker-less needle tracking, suggesting potential for clinically relevant accuracy, reliability, and reduced complexity while exhibiting robustness to needle deflection and varying needle appearances. These preliminary results indicate promising directions for future clinical development.