The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates.

Efficient interaction with the visual world requires not only object identification but also localization of where objects are in space. While spatial ("where") processing has classically been attributed to dorsal stream pathways, recent work has shown that object position can also be decoded from ventral stream responses, including the inferior temporal (IT) cortex. However, because object position in these paradigms is coupled to pixel-based location, it has remained unclear whether ventral stream position signals are perceptually meaningful or instead reflect incidental inheritance from retinotopic inputs. Here, we address this question by leveraging a visual illusion, the motion aftereffect, to dissociate perceived object position from retinal location while holding visual input constant. Combining intracortical recordings in macaque IT with matched human psychophysics, we show that motion adaptation induces direction-opponent biases in IT population codes for object position that mirror human perceptual reports, despite unchanged pixel-level input. Motion adaptation reshapes IT representational geometry that likely contributes to these perceptual biases. Extending these findings to artificial vision systems, we observe that feedforward, recurrent, and state-of-the-art video-based neural networks fail to exhibit adaptation-induced position shifts, despite accurately encoding object position. Interestingly, imposing empirically derived IT-based transformations on model features is sufficient to simulate the effect, revealing adaptation-driven representational warping as a missing computational ingredient in artificial vision systems. Together, these results identify IT as a candidate locus of perceptually aligned spatial coding, reveal adaptation-driven representational restructuring as a mechanism linking neural dynamics to perception, and expose a principled gap between biological and artificial vision.