Development and preliminary inter-rater reliability of the new PROOF tool to measure fidelity of problem-solving therapy for depression delivered by non-specialists in a low-resource African setting.

Problem-solving therapy (PST) is a brief psychological intervention often implemented for depression. Currently, there are no tools with well-evidenced reliability to measure PST fidelity. This pilot study aimed to measure the inter-rater reliability and agreement of the Problem-Solving Therapy Fidelity (PROOF) scale, comprising binary 14-item adherence and an 8-item competence subscales. Transcripts were from the TENDAI trial, a Zimbabwe-based PST intervention for depression and medication adherence. Seven transcripts were each rated by seven specialists, and two transcripts were each rated by two non-specialists. Inter-rater agreement was assessed using percent agreement and inter-rater reliability was assessed using Gwet's AC1. The PROOF subscales demonstrated promising inter-rater agreement among specialists (adherence = 90.4%, competence = 82.5%) and non-specialists (adherence = 92.9%, competence = 68.8%). Inter-rater reliability analyses yielded a Gwet's AC1 of 0.411-0.778 and 0.619-0.959 for adherence and competence among specialists, and 0.529-1.00 for adherence in non-specialists. The PROOF scale has the potential to fill the gap of fidelity tools for PST delivery.