Is statistical evaluation sufficient? New external quality assessment performance metrics through clinical performance.

OBJECTIVES

External Quality Assessment (EQA) programmes conventionally use the Standard Deviation Index (SDI) with an analyte-independent acceptance criterion of -2<SDI<+2, which does not account for biological variation or clinical relevance. We developed two novel metrics, the biologically calibrated tolerance index (SDIRCV) and percentage SDI (pSDI), that integrate biological variation into EQA interpretation and quantified the sources of analytical imprecision through variance component analysis (VCA).

METHODS

Monthly EQA results from 12 consecutive survey cycles were analysed for 45 analytes (26 biochemistry, 19 immunoassay) in a single clinical laboratory. SDIRCV was defined as the ratio of the reference change value (RCV) to the peer-group coefficient of variation. The pSDI expressed EQA bias as a percentage of the RCV. A five-zone concordance framework was constructed by cross-tabulating conventional SDI and pSDI criteria. A three-level VCA decomposed total imprecision into within-month repeatability, between-month variation, and reagent lot-to-lot variation.

RESULTS

Of 540 observations, nine (1.67 %) met the Biologically Significant, Statistically Undetected Deviation criterion (-2<SDI<+2 yet pSDI≥+100 % or ≤-100 %), indicating clinically significant bias despite acceptable SDI. Four observations (0.74 %) showed Clinically Inconsequential SDI Exceedance (SDI≥+3 or ≤-3 yet -100 %<pSDI<+100 %). Eleven analytes (24 %) had median SDIRCV<2.0, defining a persistent vulnerability zone. Lot-to-lot variation accounted for a median of 23.6 % of the total variance, exceeding 50 % for six analytes.

CONCLUSIONS

SDIRCV and pSDI detect both false acceptance and false rejection inherent in conventional EQA criteria, offering a biologically grounded framework for aligning laboratory quality assessment with clinical needs.