Complementary therapies in medicine · 2026
Inter-rater reliability and usability of CATHIS core for homeopathic intervention studies.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Loef Martin, Weiermayer Petra, Gaertner Katharina, Wrzałko Daniel, Dutta Abhijit, Saha Subhranil, Lakshani J A D S, Baumgartner Stephan, van Haselen Robbert
Аннотация
BACKGROUND: The Critical Appraisal Tool for Homeopathic Intervention Studies (CATHIS) core is a streamlined appraisal tool for homeopathic intervention studies focusing on credibility, coherence, and clinical relevance. The aim of the research project was to evaluate its inter-rater reliability, feasibility, and face validity.
METHODS: In a preregistered cross-sectional study, four raters independently applied CATHIS core to 28 trials (21 randomised controlled trials, 7 non-randomised studies on interventions) drawn from reviews on insomnia and hypertension; two external reviewers provided consensus ratings. Inter-rater reliability (IRR) was estimated using percent agreement, Fleiss' κ, and Gwet's AC2 (95% CIs). Feasibility was quantified as rating time and consensus time. Associations among the three domains were explored with correlation analyses and sensitivity checks.
RESULTS: IRR varied markedly by domain. Credibility showed good agreement (Fleiss' κ=0.66, 95% CI 0.57-0.74; AC2 =0.76, 0.71-0.82). Coherence yielded only poor-to-fair agreement (κ=0.28, 0.16-0.40; AC2 =0.41, 0.30-0.51). Clinical relevance was similarly limited (κ=0.32, 0.23-0.41; AC2 =0.36, 0.28-0.44). Individual ratings required on average 65.8 min, while consensus discussions averaged 17.7 min. Correlation analyses indicated heterogeneous and partly overlapping domain signals with limited interpretability. Face-validity responses reflected moderate-to-high acceptance but difficulties in consistent application.
CONCLUSION: CATHIS core yielded reproducible credibility ratings but only fair and operationally fragile agreement for coherence and clinical relevance, alongside non-trivial rating burden. Taken together, the current reliability profile is insufficient for confident use in systematic reviews. Targeted refinement appears warranted before broader implementation.
METHODS: In a preregistered cross-sectional study, four raters independently applied CATHIS core to 28 trials (21 randomised controlled trials, 7 non-randomised studies on interventions) drawn from reviews on insomnia and hypertension; two external reviewers provided consensus ratings. Inter-rater reliability (IRR) was estimated using percent agreement, Fleiss' κ, and Gwet's AC2 (95% CIs). Feasibility was quantified as rating time and consensus time. Associations among the three domains were explored with correlation analyses and sensitivity checks.
RESULTS: IRR varied markedly by domain. Credibility showed good agreement (Fleiss' κ=0.66, 95% CI 0.57-0.74; AC2 =0.76, 0.71-0.82). Coherence yielded only poor-to-fair agreement (κ=0.28, 0.16-0.40; AC2 =0.41, 0.30-0.51). Clinical relevance was similarly limited (κ=0.32, 0.23-0.41; AC2 =0.36, 0.28-0.44). Individual ratings required on average 65.8 min, while consensus discussions averaged 17.7 min. Correlation analyses indicated heterogeneous and partly overlapping domain signals with limited interpretability. Face-validity responses reflected moderate-to-high acceptance but difficulties in consistent application.
CONCLUSION: CATHIS core yielded reproducible credibility ratings but only fair and operationally fragile agreement for coherence and clinical relevance, alongside non-trivial rating burden. Taken together, the current reliability profile is insufficient for confident use in systematic reviews. Targeted refinement appears warranted before broader implementation.