Plastic and reconstructive surgery · 2016
Crowdsourcing as a Novel Method to Evaluate Aesthetic Outcomes of Treatment for Unilateral Cleft Lip.
Сравнительное исследованиеОценочное исследованиеСтатьяVideo-Audio MediaНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Tse Raymond W, Oh Eugene, Gruss Joseph S, Hopper Richard A, Birgfeld Craig B
Аннотация
BACKGROUND: Lack of convenient and reliable methods to grade aesthetic outcomes limits the ability to study results and optimize treatment of unilateral cleft lip. Crowdsourcing methods solicit contributions from a large group to achieve a greater task. The authors hypothesized that crowdsourcing could be used to reliably grade aesthetic outcomes of unilateral cleft lip.
METHODS: Fifty deidentified photographs of 8- to 10-year-old subjects (46 with unilateral cleft lip and four controls) were assembled. Outcomes were assessed using multiple pairwise comparisons that produced a rank order (Elo rank) of nasal appearance and, on a separate survey, by Asher-McDade ratings. Both surveys were repeated to assess reliability. A group of expert surgeons repeated the same tasks on a smaller subset of photographs.
RESULTS: The authors obtained 2500 and 1900 anonymous, layperson evaluations by means of crowdsourcing on each Elo rank and Asher-McDade survey, respectively. Elo rank and Asher-McDade scores were highly reproducible (correlation coefficients, 0.87 and 0.98), and crowd evaluations agreed with those by expert surgeons (0.980 and 0.96 for Elo rank and Asher-McDade score, respectively). Crowdsourcing surveys were completed within 9 hours, whereas the expert surgeons required 3 months. On further analysis of their cleft subject sample set, the authors found that greater initial cleft severity was associated with worse aesthetic outcome.
CONCLUSIONS: Outcomes assessed by crowds were reliable and correlated well with expert assessments. Crowdsourcing allows acquisition of massive numbers of layperson assessments on an unprecedented scale, and is a convenient, rapid, and reliable means of assessing aesthetic outcome of treatment for unilateral cleft lip.
CLINICAL QUESTION/LEVEL OF EVIDENCE: Diagnostic, IV.
METHODS: Fifty deidentified photographs of 8- to 10-year-old subjects (46 with unilateral cleft lip and four controls) were assembled. Outcomes were assessed using multiple pairwise comparisons that produced a rank order (Elo rank) of nasal appearance and, on a separate survey, by Asher-McDade ratings. Both surveys were repeated to assess reliability. A group of expert surgeons repeated the same tasks on a smaller subset of photographs.
RESULTS: The authors obtained 2500 and 1900 anonymous, layperson evaluations by means of crowdsourcing on each Elo rank and Asher-McDade survey, respectively. Elo rank and Asher-McDade scores were highly reproducible (correlation coefficients, 0.87 and 0.98), and crowd evaluations agreed with those by expert surgeons (0.980 and 0.96 for Elo rank and Asher-McDade score, respectively). Crowdsourcing surveys were completed within 9 hours, whereas the expert surgeons required 3 months. On further analysis of their cleft subject sample set, the authors found that greater initial cleft severity was associated with worse aesthetic outcome.
CONCLUSIONS: Outcomes assessed by crowds were reliable and correlated well with expert assessments. Crowdsourcing allows acquisition of massive numbers of layperson assessments on an unprecedented scale, and is a convenient, rapid, and reliable means of assessing aesthetic outcome of treatment for unilateral cleft lip.
CLINICAL QUESTION/LEVEL OF EVIDENCE: Diagnostic, IV.