DOI 10.1097/prs.0000000000002545
Crowdsourcing as a Novel Method to Evaluate Aesthetic Outcomes of Treatment for Unilateral Cleft Lip.
Аннотация
BACKGROUND
Lack of convenient and reliable methods to grade aesthetic outcomes limits the ability to study results and optimize treatment of unilateral cleft lip. Crowdsourcing methods solicit contributions from a large group to achieve a greater task. The authors hypothesized that crowdsourcing could be used to reliably grade aesthetic outcomes of unilateral cleft lip.
METHODS
Fifty deidentified photographs of 8- to 10-year-old subjects (46 with unilateral cleft lip and four controls) were assembled. Outcomes were assessed using multiple pairwise comparisons that produced a rank order (Elo rank) of nasal appearance and, on a separate survey, by Asher-McDade ratings. Both surveys were repeated to assess reliability. A group of expert surgeons repeated the same tasks on a smaller subset of photographs.
RESULTS
The authors obtained 2500 and 1900 anonymous, layperson evaluations by means of crowdsourcing on each Elo rank and Asher-McDade survey, respectively. Elo rank and Asher-McDade scores were highly reproducible (correlation coefficients, 0.87 and 0.98), and crowd evaluations agreed with those by expert surgeons (0.980 and 0.96 for Elo rank and Asher-McDade score, respectively). Crowdsourcing surveys were completed within 9 hours, whereas the expert surgeons required 3 months. On further analysis of their cleft subject sample set, the authors found that greater initial cleft severity was associated with worse aesthetic outcome.
CONCLUSIONS
Outcomes assessed by crowds were reliable and correlated well with expert assessments. Crowdsourcing allows acquisition of massive numbers of layperson assessments on an unprecedented scale, and is a convenient, rapid, and reliable means of assessing aesthetic outcome of treatment for unilateral cleft lip.
CLINICAL QUESTION/LEVEL OF EVIDENCE: Diagnostic, IV.
Предметные рубрики MeSH
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.