Frontiers in digital health · 2026
Automated emotion recognition via video-based semantic embeddings.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Diemerling Hannes, Kulla Patricia, Kruse Joachim, von Oertzen Timo
Аннотация
INTRODUCTION: Automated emotion recognition systems often rely on acted datasets and categorical models that miss the nuance of spontaneous affect.
METHODS: This work assembled a large corpus of authentic facial emotion expressions from naturalistic outpatient psychotherapy sessions, annotated with free-text descriptions by human labelers. These descriptions were embedded in a 768-dimensional semantic space using a fine-tuned German Sentence-BERT model. Transformer, BILSTM, and deep neural network architectures were trained to map facial landmark features to continuous emotion embeddings.
RESULTS: Leave-one-out cross-validation showed model predictions closely matched human annotations with a mean z-score of 1.97. External evaluation against acted datasets (RAVDESS) confirmed strong recognition of joy, sadness, and fear.
DISCUSSION: To enhance interpretability, a back-translation mechanism using cosine similarity was implemented and visualized with radar charts. All components were integrated into AFFECT, an open-source pipeline for analyzing emotional expressions in everyday video recordings.
METHODS: This work assembled a large corpus of authentic facial emotion expressions from naturalistic outpatient psychotherapy sessions, annotated with free-text descriptions by human labelers. These descriptions were embedded in a 768-dimensional semantic space using a fine-tuned German Sentence-BERT model. Transformer, BILSTM, and deep neural network architectures were trained to map facial landmark features to continuous emotion embeddings.
RESULTS: Leave-one-out cross-validation showed model predictions closely matched human annotations with a mean z-score of 1.97. External evaluation against acted datasets (RAVDESS) confirmed strong recognition of joy, sadness, and fear.
DISCUSSION: To enhance interpretability, a back-translation mechanism using cosine similarity was implemented and visualized with radar charts. All components were integrated into AFFECT, an open-source pipeline for analyzing emotional expressions in everyday video recordings.