Landmark-free automatic digital twin registration in robot-assisted partial nephrectomy using a generic end-to-end model.

PURPOSE: Augmented Reality in Minimally Invasive Surgery has made tremendous progress in organs including the liver and the uterus. The core problem of Augmented Reality is registration, where a preoperative patient's geometric digital twin must be aligned with the image of the surgical camera. The case of the kidney is yet unresolved, owing to the absence of anatomical landmarks visible in both the patient's digital twin and the surgical images.

METHODS

We propose a landmark-free approach to registration, which is particularly well-adapted to the kidney. The approach involves a generic kidney model and an end-to-end neural network, which we train with a proposed dataset to regress the registration directly from a surgical RGB image.

RESULTS

Experimental evaluation across four clinical cases demonstrates strong concordance with expert-labelled registration, despite anatomical and motion variability. The proposed method achieved an average tumour contour alignment error of 7.3 ± 4.1 mm in 9.4 ± 0.2 ms.

CONCLUSION

This landmark-free registration approach meets the accuracy, speed and resource constraints required in clinical practice, making it a promising tool for Augmented Reality-Assisted Partial Nephrectomy.