Quantum spin nematic phase in a square-lattice iridate.

Spin nematic is a magnetic analogue of classical liquid crystals, a fourth state of matter exhibiting characteristics of both liquid and solid1,2. Particularly intriguing is a valence-bond spin nematic3-5, in which spins are quantum entangled to form a multipolar order without breaking time-reversal symmetry, but its unambiguous experimental realization remains elusive. Here we establish a spin nematic phase in the square-lattice iridate Sr2IrO4, which approximately realizes a pseudospin one-half Heisenberg antiferromagnet in the strong spin-orbit coupling limit6-9. Upon cooling, the transition into the spin nematic phase at TC ≈ 263 K is marked by a divergence in the static spin quadrupole susceptibility extracted from our Raman spectra and concomitant emergence of a collective mode associated with the spontaneous breaking of rotational symmetries. The quadrupolar order persists in the antiferromagnetic phase below TN ≈ 230 K and becomes directly observable through its interference with the antiferromagnetic order in resonant X-ray diffraction, which allows us to uniquely determine its spatial structure. Further, we find using resonant inelastic X-ray scattering a complete breakdown of coherent magnon excitations at short-wavelength scales, suggesting a many-body quantum entanglement in the antiferromagnetic state10,11. Taken together, our results reveal a quantum order underlying the Néel antiferromagnet that is widely believed to be intimately connected to the mechanism of high-temperature superconductivity12,13.