Designing Studies to Ensure that Estimates of Test Accuracy Will Travel

There may be genuine differences between test accuracies in different settings, such as primary care or hospital, in different types of hospital, or between countries. Deciding whether estimates of test accuracy are transferable to other settings depends on an understanding of the possible reasons for variability in test discrimination and calibration across settings. The transferability of measures of test performance from one setting to another depends on which indicator of test performance is to be used. Real variation in the performance of diagnostic tests (such as different test types, or a different spectrum of disease) needs to be distinguished from artifactual variation resulting from study design features. These features include the target condition and reference standard used, the population and the clinical question studied, the evaluated comparison, and the way the index test was performed, calibrated, and interpreted. In preparing studies on diagnostic accuracy, a key question is how to design studies that carry more information about the transferability of results. To ensure that estimates of diagnostic accuracy will travel, before starting to design a study the following questions must be answered: How are the target condition and reference standard defined? - Is the objective to estimate global test performance or to estimate probability of disease in individuals? - What is the population and clinical problem? - Is the test being considered as a replacement or incremental test? - To what extent do you want to study the reasons for variability of the results within your population? - To what extent do you want to study the transferability of the results to other settings? Designing studies with heterogeneous study populations allows exploration of the transferability of diagnostic performance in different settings. This will require larger studies than have generally been carried out in the past for diagnostic tests.