TY - JOUR
T1 - The bumpy road to achieve reliability of clinical profile characteristics in psychosis and related disorders
AU - Berendsen, Steven
AU - van Tricht, Mirjam J.
AU - Tedja, Amy
AU - Burger, Thijs J.
AU - de Koning, Mariken B.
AU - de Haan, Lieuwe
N1 - Publisher Copyright: © 2020 The Authors. International Journal of Methods in Psychiatric Research published by John Wiley & Sons Ltd. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021/6/1
Y1 - 2021/6/1
N2 - Objectives: Profile characteristics are factors that are relevant for diagnosis, prognosis or treatment. The present study aims to develop a set of clinically relevant profile characteristics. Moreover, our goal is to determine the inter-rater reliability (IRR) of the selected profile characteristics. Methods: Potential profile characteristics were determined by literature review. Assessment of IRR was done by comparing scores on profile characteristics determined by two researchers. We conducted three subsequent studies: (1) assessment of pre-training IRR, (2) IRR following implementation of an instruction manual, (3) IRR after optimizing scoring methods. IRR was measured with the Intraclass Correlation Coefficient (ICC). Results: IRR scores of profile characteristic Illegal activities were high across the three studies (ICC ≥ 0.75). Following training procedures in study 2 and 3, reliability estimates remained low to moderate (ICC < 0.75) for the profile characteristics Support of relatives, Aggression recent and lifetime, substance use and insight recent. IRR scores of the other eight profile characteristics varied from low, moderate to high across studies. Conclusion: IRR scores of profile characteristics were highly variable, and mostly inadequate in all three studies. Consequently, further research should focus on specification of severity scores of profile characteristics, optimizing scoring methods and re-evaluation of IRR.
AB - Objectives: Profile characteristics are factors that are relevant for diagnosis, prognosis or treatment. The present study aims to develop a set of clinically relevant profile characteristics. Moreover, our goal is to determine the inter-rater reliability (IRR) of the selected profile characteristics. Methods: Potential profile characteristics were determined by literature review. Assessment of IRR was done by comparing scores on profile characteristics determined by two researchers. We conducted three subsequent studies: (1) assessment of pre-training IRR, (2) IRR following implementation of an instruction manual, (3) IRR after optimizing scoring methods. IRR was measured with the Intraclass Correlation Coefficient (ICC). Results: IRR scores of profile characteristic Illegal activities were high across the three studies (ICC ≥ 0.75). Following training procedures in study 2 and 3, reliability estimates remained low to moderate (ICC < 0.75) for the profile characteristics Support of relatives, Aggression recent and lifetime, substance use and insight recent. IRR scores of the other eight profile characteristics varied from low, moderate to high across studies. Conclusion: IRR scores of profile characteristics were highly variable, and mostly inadequate in all three studies. Consequently, further research should focus on specification of severity scores of profile characteristics, optimizing scoring methods and re-evaluation of IRR.
KW - clinical profiling
KW - heterogeneity
KW - inter-rater reliability
KW - psychosis
KW - training procedures
UR - http://www.scopus.com/inward/record.url?scp=85097290902&partnerID=8YFLogxK
U2 - https://doi.org/10.1002/mpr.1858
DO - https://doi.org/10.1002/mpr.1858
M3 - Article
C2 - 33615618
SN - 1049-8931
VL - 30
JO - International journal of methods in psychiatric research
JF - International journal of methods in psychiatric research
IS - 2
M1 - e1858
ER -