Schizophrenia bulletin · 2022
Natural Language Processing and Psychosis: On the Need for Comprehensive Psychometric Evaluation.
СтатьяПоддержка NIH: внешняя программаНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Cohen Alex S, Rodriguez Zachary, Warren Kiara K, Cowan Tovah, Masucci Michael D, Edvard Granrud Ole, Holmlund Terje B, Chandler Chelsea, Foltz Peter W, Strauss Gregory P
Аннотация
BACKGROUND AND HYPOTHESIS: Despite decades of "proof of concept" findings supporting the use of Natural Language Processing (NLP) in psychosis research, clinical implementation has been slow. One obstacle reflects the lack of comprehensive psychometric evaluation of these measures. There is overwhelming evidence that criterion and content validity can be achieved for many purposes, particularly using machine learning procedures. However, there has been very little evaluation of test-retest reliability, divergent validity (sufficient to address concerns of a "generalized deficit"), and potential biases from demographics and other individual differences.
STUDY DESIGN: This article highlights these concerns in development of an NLP measure for tracking clinically rated paranoia from video "selfies" recorded from smartphone devices. Patients with schizophrenia or bipolar disorder were recruited and tracked over a week-long epoch. A small NLP-based feature set from 499 language samples were modeled on clinically rated paranoia using regularized regression.
STUDY RESULTS: While test-retest reliability was high, criterion, and convergent/divergent validity were only achieved when considering moderating variables, notably whether a patient was away from home, around strangers, or alone at the time of the recording. Moreover, there were systematic racial and sex biases in the model, in part, reflecting whether patients submitted videos when they were away from home, around strangers, or alone.
CONCLUSIONS: Advancing NLP measures for psychosis will require deliberate consideration of test-retest reliability, divergent validity, systematic biases and the potential role of moderators. In our example, a comprehensive psychometric evaluation revealed clear strengths and weaknesses that can be systematically addressed in future research.
STUDY DESIGN: This article highlights these concerns in development of an NLP measure for tracking clinically rated paranoia from video "selfies" recorded from smartphone devices. Patients with schizophrenia or bipolar disorder were recruited and tracked over a week-long epoch. A small NLP-based feature set from 499 language samples were modeled on clinically rated paranoia using regularized regression.
STUDY RESULTS: While test-retest reliability was high, criterion, and convergent/divergent validity were only achieved when considering moderating variables, notably whether a patient was away from home, around strangers, or alone at the time of the recording. Moreover, there were systematic racial and sex biases in the model, in part, reflecting whether patients submitted videos when they were away from home, around strangers, or alone.
CONCLUSIONS: Advancing NLP measures for psychosis will require deliberate consideration of test-retest reliability, divergent validity, systematic biases and the potential role of moderators. In our example, a comprehensive psychometric evaluation revealed clear strengths and weaknesses that can be systematically addressed in future research.