Comprehensive RGBD-based shoulder kinematics using markerless anatomical landmarks detection and a 3D-printed acromial cluster during hand-cycling.

Biomechanical biofeedback has the potential to enhance rehabilitation by providing clinicians with objective evaluation of patient performances. As feedback systems often depend on expensive and sophisticated motion capture technologies, researchers explore computer vision-based alternatives. Existing methods suffer from substantial joint angle errors, particularly in the upper limb, and neglect the scapular movements. In this paper, we present a method based on a single front-facing RGB-D camera that automatically detects 3D anatomical landmark locations using depth information. We also use a 3D-printed acromial cluster to provide scapular motion. Together, these landmarks and the acromial cluster are used to provide comprehensive estimation of shoulder joint kinematics through inverse kinematics. Annotated images from eight participants were used to fine-tune a convolutional neural network, which was subsequently evaluated on a hand-cycling motion. Our method showed a strong agreement with a reference marker-based system, with 3D anatomical landmark detection errors averaging 5 mm. The resulting kinematics closely aligned with the reference system, maintaining acceptable joint angle errors (∼6.3°). Furthermore, the algorithm could provide real-time anatomical landmark positions and joint kinematics at a rate of 50Hz. This study highlights the potential of using a single consumer-grade depth-sensing camera combined with a 3D-printed acromial cluster to accurately estimate upper-limb kinematics through anatomical landmark detection, paving the way for more accessible clinical assessments.