Global clinical performance rating, reliability and validity in an undergraduate clerkship

H. E.M. Daelmans, H. H. van der Hem-Stokroos, R. J.I. Hoogenboom, A. J.J.A. Scherpbier, C. D.A. Stehouwer, C. P.M. van der Vleuten

Research output: Contribution to journalArticleAcademicpeer-review

16 Citations (Scopus)

Abstract

Background: Global performance rating is frequently used in clinical training despite its known psychometric drawbacks. Inter-rater reliability is low in undergraduate training but better in residency training, possibly because residency offers more opportunities for supervision. The low or moderate predictive validity of global performance ratings in undergraduate and residency training may be due to low or unknown reliability of both global performance ratings and criterion measures. In an undergraduate clerkship, we investigated whether reliability improves when raters are more familiar with students' work and whether validity improves with increased reliability of the predictor and criterion instrument. Methods: Inter-rater reliability was determined in a clerkship with more student-rater contacts than usual. The in-training assessment programme of the clerkship that immediately followed was used as the criterion measure to determine predictive validity. Results: With four ratings, inter-rater reliability was 0.41 and predictive validity was 0.32. Reliability was lower and validity slightly higher than similar results published for residency training. Conclusion: Even with increased student-rater interaction, the reliability and validity of global performance ratings were too low to warrant the usage of global performance ratings as individual assessment format. However, combined with other assessment measures, global performance ratings may lead to improved integral assessment.

Original languageEnglish
Pages (from-to)279-284
Number of pages6
JournalNetherlands journal of medicine
Volume63
Issue number7
Publication statusPublished - Jul 2005

Keywords

  • Clerkship
  • Disattenuation
  • Global clinical performance rating
  • Inter-rater reliability
  • Predictive validity

Cite this