TY - JOUR
T1 - Unlike ROC analysis, a new IRT method identified clinical thresholds unbiased by disease prevalence
AU - Terluin, Berend
AU - Griffiths, Philip
AU - van der Wouden, Johannes C.
AU - Ingelsrud, Lina Holm
AU - Terwee, Caroline B.
PY - 2020/8
Y1 - 2020/8
N2 - Objective: This study introduces a new method to establish clinical thresholds for multi-item tests, based on item response theory (IRT), as an alternative to receiver operating characteristic (ROC) analysis. The performance of IRT method was examined and compared with the ROC method across multiple simulated data sets and in a real data set. Study Design and Setting: Simulated data sets (sample size: 1,000) varied in means and variability of the test scores and the prevalence of disease. The true clinical threshold was defined as a predetermined location on the latent trait underlying the questionnaire, with its corresponding expected test score. The real data set (sample size: 295) comprised Hospital Anxiety Depression Scale (HADS) depression scores and Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition major depressive disorder (MDD) diagnoses. Results: The IRT method recovered the clinical thresholds without bias, whereas the ROC method identified thresholds that were biased by the prevalence of disease. Mild MDD was clinically diagnosed in 23%, moderate MDD in 12%, and severe MDD in 14% of the participants. The IRT method identified the following HADS depression score thresholds for mild, moderate, and severe MDD: 10.7, 13.2, and 15.1, respectively. Conclusion: The new IRT method identifies clinical thresholds that are unbiased by disease prevalence.
AB - Objective: This study introduces a new method to establish clinical thresholds for multi-item tests, based on item response theory (IRT), as an alternative to receiver operating characteristic (ROC) analysis. The performance of IRT method was examined and compared with the ROC method across multiple simulated data sets and in a real data set. Study Design and Setting: Simulated data sets (sample size: 1,000) varied in means and variability of the test scores and the prevalence of disease. The true clinical threshold was defined as a predetermined location on the latent trait underlying the questionnaire, with its corresponding expected test score. The real data set (sample size: 295) comprised Hospital Anxiety Depression Scale (HADS) depression scores and Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition major depressive disorder (MDD) diagnoses. Results: The IRT method recovered the clinical thresholds without bias, whereas the ROC method identified thresholds that were biased by the prevalence of disease. Mild MDD was clinically diagnosed in 23%, moderate MDD in 12%, and severe MDD in 14% of the participants. The IRT method identified the following HADS depression score thresholds for mild, moderate, and severe MDD: 10.7, 13.2, and 15.1, respectively. Conclusion: The new IRT method identifies clinical thresholds that are unbiased by disease prevalence.
KW - Clinical threshold
KW - Cutoff score
KW - Diagnostic threshold
KW - Item response theory
KW - Receiver operating characteristic analysis
UR - http://www.scopus.com/inward/record.url?scp=85085554092&partnerID=8YFLogxK
U2 - https://doi.org/10.1016/j.jclinepi.2020.05.008
DO - https://doi.org/10.1016/j.jclinepi.2020.05.008
M3 - Article
C2 - 32438022
SN - 0895-4356
VL - 124
SP - 118
EP - 125
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
ER -