Sampling of natural speech for the assessment of psychopathology: data collection procedure and inter-rater reliability.

OBJECTIVES

Recent research explores speech as a marker of psychiatric illness. Applications of speech analysis in clinical settings depend on reliable methods of speech data collection and processing, yet no standard exists thus far. We present a speech sampling and coding procedure developed to standardize methods across literature and facilitate assessment of psychiatric features through speech.

METHODS

We developed a procedure to collect naturalistic speech samples from adults using emotionally-valenced prompting. Speech variables characterize features of illness that can aid in assessment of psychopathology. The samples are transcribed, segmented, and coded by human raters on variables indexing speaker, reference, emotions, and features of psychopathology.

RESULTS

Samples from 200 adult participants were analyzed. The protocol elicited an average of 10 minutes of speech per assessment. Almost all participants provided 5 minutes or more of speech suitable for analysis. Human coders identified segment-level sentiment, reference, and emotions with high inter-rater reliability (ICC 0.60 to 0.79), but agreed less consistently on more complex features, such as worry and rumination (ICC 0.24 to 0.47).

CONCLUSION

We present a speech collection and processing method that elicits speech samples from participants with a range of psychopathology, suitable for manual and computerized analysis with applications to clinical settings.