A blinded study of inter-rater agreement of video signs for potential concussion in the National Rugby League comparing novice versus expert reviewers.

Inter-rater agreement between novice and expert clinicians for the six international consensus video signs of concussion was investigated in the National Rugby League. Consecutive head injury events from the 2017 and 2018 seasons were included (n = 636). Video footage was examined by two novice and two experienced raters. Agreement for any video sign and the exact combination of video signs was fair (Gwet's AC1 of 0.31, 0.37). Agreement for individual video signs ranged from fair (motor incoordination, Gwet's AC1 0.28) to almost perfect (impact seizures, Gwet's AC1 0.99). Agreement was significantly better between experienced raters compared to novices (p < 0.05). Greater educational efforts might improve the accuracy of identifying possible concussions.