Construct validation of progress testing to measure knowledge and visual skills in radiology

BACKGROUND: The Dutch Radiology Progress Test (DRPT) monitors the acquisition of knowledge and visual skills of radiology residents in the Netherlands.

AIM: We aimed to evaluate the quality of progress testing in postgraduate radiology training by studying the reliability of the DRPT and finding an indication for its construct validity. We expected that knowledge would increase rapidly in the first years of residency, leveling-off in later years, to allow for the development of visual skills. We hypothesized that scores on the DRPT reflect this pattern.

METHODS: Internal consistencies were estimated with Cronbach's alpha. Performance increase over program years were tested using one-way analysis of variance.

RESULTS: Data were available for 498 residents (2281 test results). Reliabilities were around Cronbach's alpha 0.90. There was a significant difference in the mean test results between the first three years of residency. After the fourth year no significant increase in test scores on knowledge could be measured on eight tests. The same pattern occurred for scores on visual skills. However, visual skills scores tend to increase more sharply than knowledge scores.

CONCLUSION: We found support for the reliability and construct validity of the DRPT. However, assessment on visual skill development needs further exploration.