Supporting Reliable Assessment of Clinical Judgment Skills in Simulation Environments.

BACKGROUND

Clinical judgment skills, a key component of nursing competency, are difficult to accurately and consistently assess. To support evaluation of these skills using Lasater's (2007) clinical judgment rubric (LCJR), rater training resources were developed and evaluated.

METHOD: A one-group pretest-posttest design evaluated the effect of developed resources on agreement between two novice raters (NRs) and resource developers. NR-CE had content expertise; NR-NNE had no nursing experience.

RESULTS

Pre-training, total scores showed poor agreement between novice and trained raters (ICCabsolute(2,1) LCJRtotal for NR-CE = -.59; LCJRtotal for NR-NNE = .24). Posttraining, agreement was higher (ICCabsolute(2,1) LCJRtotal NR-CE = .70; LCJRtotal NR-NNE = .76). Dimension-level scoring showed more variability in reliability estimates.

CONCLUSION

Training supported raters with and without content expertise in developing a shared mental model of CJS. Future work should replicate these findings with larger samples for more confidence in results and examine convergent validity by comparing ratings to external benchmarks.